Course 1 - Custom Models, Layers and Loss Functions/Week 1 - Functional API/W1_Assignment/C1W1_Assignment.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ZL_6GK8qX35J"
   },
   "source": [
    "\n",
    "\n",
    "# Week 1: Multiple Output Models using the Keras Functional API\n",
    "\n",
    "Welcome to the first programming assignment of the course! Your task will be to use the Keras functional API to train a model to predict two outputs. For this lab, you will use the **[Wine Quality Dataset](https://archive.ics.uci.edu/ml/datasets/Wine+Quality)** from the **UCI machine learning repository**. It has separate datasets for red wine and white wine.\n",
    "\n",
    "Normally, the wines are classified into one of the quality ratings specified in the attributes. In this exercise, you will combine the two datasets to predict the wine quality and whether the wine is red or white solely from the attributes. \n",
    "\n",
    "You will model wine quality estimations as a regression problem and wine type detection as a binary classification problem.\n",
    "\n",
    "#### Please complete sections that are marked **(TODO)**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "obdcD6urYBY9"
   },
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "t8N3pcTQ5oQI"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow.keras.models import Model\n",
    "from tensorflow.keras.layers import Dense, Input\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay\n",
    "import itertools\n",
    "\n",
    "import utils"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gQMERzWQYpgm"
   },
   "source": [
    "## Load Dataset\n",
    "\n",
    "\n",
    "You will now load the dataset from the [UCI Machine Learning Repository](https://archive.ics.uci.edu/ml/index.php) which are **already saved** in your workspace (*Note: For successful grading, please **do not** modify the default string set to the `URI` variable below*).\n",
    "\n",
    "### Pre-process the white wine dataset (TODO)\n",
    "You will add a new column named `is_red` in your dataframe to indicate if the wine is white or red. \n",
    "- In the white wine dataset, you will fill the column `is_red` with  zeros (0)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "2qYAjKXCd4RH",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "325ea195519b7035934c95bb529a062c",
     "grade": false,
     "grade_id": "cell-e5bfa0f152d9a21f",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "# URL of the white wine dataset\n",
    "URI = './winequality-white.csv'\n",
    "\n",
    "# load the dataset from the URL\n",
    "white_df = pd.read_csv(URI, sep=\";\")\n",
    "\n",
    "# fill the `is_red` column with zeros.\n",
    "white_df[\"is_red\"] = 0 # YOUR CODE HERE\n",
    "\n",
    "# keep only the first of duplicate items\n",
    "white_df = white_df.drop_duplicates(keep='first')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "defe38d6ec58fd31cd67b89e46c4373f",
     "grade": true,
     "grade_id": "cell-30575e713b55fc51",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "# You can click `File -> Open` in the menu above and open the `utils.py` file \n",
    "# in case you want to inspect the unit tests being used for each graded function.\n",
    "\n",
    "utils.test_white_df(white_df)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "OQHK0ohBQRCk"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "8.8\n",
      "9.1\n"
     ]
    }
   ],
   "source": [
    "print(white_df.alcohol[0])\n",
    "print(white_df.alcohol[100])\n",
    "\n",
    "# EXPECTED OUTPUT\n",
    "# 8.8\n",
    "# 9.1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pre-process the red wine dataset (TODO)\n",
    "- In the red wine dataset, you will fill in the column `is_red` with ones (1)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "8y3QxKwBed8v",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "12e0963d15be33b01b4e6ebc8945e51e",
     "grade": false,
     "grade_id": "cell-e47a40f306593274",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "# URL of the red wine dataset\n",
    "URI = './winequality-red.csv'\n",
    "\n",
    "# load the dataset from the URL\n",
    "red_df = pd.read_csv(URI, sep=\";\")\n",
    "\n",
    "# fill the `is_red` column with ones.\n",
    "red_df[\"is_red\"] = 1 # YOUR CODE HERE\n",
    "\n",
    "# keep only the first of duplicate items\n",
    "red_df = red_df.drop_duplicates(keep='first')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "d8e0c91b0fd668b63ba74a8f2f958b59",
     "grade": true,
     "grade_id": "cell-2a75937adcc0c25b",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_red_df(red_df)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "zsB3LUzNQpo_"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9.4\n",
      "10.2\n"
     ]
    }
   ],
   "source": [
    "print(red_df.alcohol[0])\n",
    "print(red_df.alcohol[100])\n",
    "\n",
    "# EXPECTED OUTPUT\n",
    "# 9.4\n",
    "# 10.2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2G8B-NYuM6-f"
   },
   "source": [
    "### Concatenate the datasets\n",
    "\n",
    "Next, concatenate the red and white wine dataframes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "YpQrOjJbfN3m"
   },
   "outputs": [],
   "source": [
    "df = pd.concat([red_df, white_df], ignore_index=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Se2dTmThQyjb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9.4\n",
      "9.5\n"
     ]
    }
   ],
   "source": [
    "print(df.alcohol[0])\n",
    "print(df.alcohol[100])\n",
    "\n",
    "# EXPECTED OUTPUT\n",
    "# 9.4\n",
    "# 9.5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In a real-world scenario, you should shuffle the data. For this assignment however, **you are not** going to do that because the grader needs to test with deterministic data. If you want the code to do it **after** you've gotten your grade for this notebook, we left the commented line below for reference"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "wx6y3rPpQv4k"
   },
   "outputs": [],
   "source": [
    "#df = df.iloc[np.random.permutation(len(df))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-EqIcbg5M_n1"
   },
   "source": [
    "This will chart the quality of the wines."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "IsvK0-Sgy17C"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD4CAYAAAAAczaOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAQk0lEQVR4nO3df6zddX3H8edL6rAUmShyw1q2sqQxAs1QbhgbCbmMTasYwWUmJUxgc6khuOjWZCn7xy1LE5aM/ZANsg4cNSJNh5KSIU7CdudMRCyKKT8kdFKxlFEdiJQZtPjeH/eLXtrT9vbcH+fc83k+kpNz7ud8P9/zfvec+7rf+7nfc5qqQpLUhtcMugBJ0sIx9CWpIYa+JDXE0Jekhhj6ktSQJYMu4EhOOumkWrlyZV9zX3zxRZYtWza3BQ3IqPQyKn2AvQyrUelltn088MAD36uqNx84PvShv3LlSrZv397X3MnJSSYmJua2oAEZlV5GpQ+wl2E1Kr3Mto8k3+417vKOJDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1ZOjfkSsNqx1PPc+VG+7qa+6uay+a42qkmfFIX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1JAjhn6SU5P8R5JHkzyc5CPd+BuT3JPk8e76xGlzrkmyM8ljSd45bfzsJDu6+z6eJPPTliSpl5kc6e8H1lfVW4FzgauTnA5sAO6tqlXAvd3XdPetBc4A1gA3JDmm29eNwDpgVXdZM4e9SJKO4IihX1VPV9XXutsvAI8Cy4GLgc3dZpuBS7rbFwNbquqlqnoC2Amck+QU4ISq+nJVFfDJaXMkSQvgqNb0k6wE3gZ8BRirqqdh6gcDcHK32XLgO9Om7e7Glne3DxyXJC2QJTPdMMnxwGeAj1bVDw6zHN/rjjrMeK/HWsfUMhBjY2NMTk7OtMxX2bdvX99zh82o9DIqfQCMLYX1q/f3NXfY/g1G6XkZlV7mq48ZhX6S1zIV+LdW1We74WeSnFJVT3dLN3u78d3AqdOmrwD2dOMreowfpKo2AZsAxsfHa2JiYmbdHGBycpJ+5w6bUellVPoAuP7WbVy3Y8bHTa+y67KJuS1mlkbpeRmVXuarj5mcvRPgZuDRqvrraXfdCVzR3b4C2DZtfG2SY5OcxtQfbO/vloBeSHJut8/Lp82RJC2AmRymnAd8ANiR5MFu7E+Ba4GtST4IPAm8H6CqHk6yFXiEqTN/rq6ql7t5VwG3AEuBu7uLJGmBHDH0q+pL9F6PB7jwEHM2Aht7jG8HzjyaAiVJc8d35EpSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JD+vtfnaUDrNxw14y2W796P1cesO2uay+aj5Ik9eCRviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhwx9JN8IsneJA9NG/uzJE8lebC7vHvafdck2ZnksSTvnDZ+dpId3X0fT5K5b0eSdDgzOdK/BVjTY/xvquqs7vI5gCSnA2uBM7o5NyQ5ptv+RmAdsKq79NqnJGkeHTH0q+qLwLMz3N/FwJaqeqmqngB2AuckOQU4oaq+XFUFfBK4pN+iJUn9WTKLuR9OcjmwHVhfVc8By4H7pm2zuxv7cXf7wPGekqxj6rcCxsbGmJyc7KvAffv29T132Ax7L+tX75/RdmNLD952mPs6nF69zNSw9Tzsr6+jMSq9zFcf/Yb+jcBfANVdXwf8PtBrnb4OM95TVW0CNgGMj4/XxMREX0VOTk7S79xhM+y9XLnhrhltt371fq7b8eqX3a7LJuahovl3/a3bDuplpoat52F/fR2NUellvvro6+ydqnqmql6uqp8A/wSc0921Gzh12qYrgD3d+Ioe45KkBdRX6Hdr9K94H/DKmT13AmuTHJvkNKb+YHt/VT0NvJDk3O6sncuBbbOoW5LUhyP+bprkNmACOCnJbuBjwESSs5haotkFfAigqh5OshV4BNgPXF1VL3e7uoqpM4GWAnd3F0nSAjpi6FfVpT2Gbz7M9huBjT3GtwNnHlV1kqQ55TtyJakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhwx9JN8IsneJA9NG3tjknuSPN5dnzjtvmuS7EzyWJJ3Ths/O8mO7r6PJ8nctyNJOpyZHOnfAqw5YGwDcG9VrQLu7b4myenAWuCMbs4NSY7p5twIrANWdZcD9ylJmmdHDP2q+iLw7AHDFwObu9ubgUumjW+pqpeq6glgJ3BOklOAE6rqy1VVwCenzZEkLZB+1/THquppgO765G58OfCdadvt7saWd7cPHJckLaAlc7y/Xuv0dZjx3jtJ1jG1FMTY2BiTk5N9FbNv376+5w6bYe9l/er9M9pubOnB2w5zX4fTq5eZGraeh/31dTRGpZf56qPf0H8mySlV9XS3dLO3G98NnDptuxXAnm58RY/xnqpqE7AJYHx8vCYmJvoqcnJykn7nDpth7+XKDXfNaLv1q/dz3Y5Xv+x2XTYxDxXNv+tv3XZQLzM1bD0P++vraIxKL/PVR7/LO3cCV3S3rwC2TRtfm+TYJKcx9Qfb+7sloBeSnNudtXP5tDmSpAVyxMOUJLcBE8BJSXYDHwOuBbYm+SDwJPB+gKp6OMlW4BFgP3B1Vb3c7eoqps4EWgrc3V0kSQvoiKFfVZce4q4LD7H9RmBjj/HtwJlHVZ0kaU75jlxJaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhoy15+9I2nIrezxkRnrV++f0Udp7Lr2ovkoSQvII31JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1ZFahn2RXkh1JHkyyvRt7Y5J7kjzeXZ84bftrkuxM8liSd862eEnS0ZmLI/0Lquqsqhrvvt4A3FtVq4B7u69JcjqwFjgDWAPckOSYOXh8SdIMzcfyzsXA5u72ZuCSaeNbquqlqnoC2AmcMw+PL0k6hFRV/5OTJ4DngAL+sao2Jfl+Vb1h2jbPVdWJSf4euK+qPtWN3wzcXVW399jvOmAdwNjY2Nlbtmzpq759+/Zx/PHH9zV32Ax7Lzueen5G240thWd++Oqx1ct/fh4qmn97n33+oF5mapA993quej0vvSyG52rYv1dmarZ9XHDBBQ9MW4H5qSWzqgrOq6o9SU4G7knyzcNsmx5jPX/iVNUmYBPA+Ph4TUxM9FXc5OQk/c4dNsPey5Ub7prRdutX7+e6Ha9+2e26bGIeKpp/19+67aBeZmqQPfd6rno9L70shudq2L9XZmq++pjV8k5V7emu9wJ3MLVc80ySUwC6673d5ruBU6dNXwHsmc3jS5KOTt+hn2RZkte/cht4B/AQcCdwRbfZFcC27vadwNokxyY5DVgF3N/v40uSjt5slnfGgDuSvLKfT1fV55N8Fdia5IPAk8D7Aarq4SRbgUeA/cDVVfXyrKqXJB2VvkO/qr4F/EqP8f8FLjzEnI3Axn4fU5I0O74jV5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDen7P0bX/Fm54a6Dxtav3s+VPcYPtOvai+ajJEkjwiN9SWqIoS9JDTH0Jakhhr4kNcTQl6SGePaOpKHX64y2Q+l1pptntf2MR/qS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktSQBX9HbpI1wN8BxwA3VdW18/VYO556fkafQd+L7+CTNIoW9Eg/yTHAPwDvAk4HLk1y+kLWIEktW+gj/XOAnVX1LYAkW4CLgUcWuA5JmndH85lBB7plzbI5rORnUlXzsuOeD5b8DrCmqv6g+/oDwK9W1YcP2G4dsK778i3AY30+5EnA9/qcO2xGpZdR6QPsZViNSi+z7eOXqurNBw4u9JF+eowd9FOnqjYBm2b9YMn2qhqf7X6Gwaj0Mip9gL0Mq1HpZb76WOizd3YDp077egWwZ4FrkKRmLXTofxVYleS0JD8HrAXuXOAaJKlZC7q8U1X7k3wY+DemTtn8RFU9PI8POesloiEyKr2MSh9gL8NqVHqZlz4W9A+5kqTB8h25ktQQQ1+SGjJyoZ/kdUnuT/KNJA8n+fNB1zRbSY5J8vUk/zroWmYjya4kO5I8mGT7oOuZjSRvSHJ7km8meTTJrw26pqOV5C3dc/HK5QdJPjrouvqV5I+67/mHktyW5HWDrqlfST7S9fHwXD8nI7emnyTAsqral+S1wJeAj1TVfQMurW9J/hgYB06oqvcMup5+JdkFjFfVon/jTJLNwH9V1U3dmWjHVdX3B11Xv7qPSHmKqTdLfnvQ9RytJMuZ+l4/vap+mGQr8LmqumWwlR29JGcCW5j6BIMfAZ8Hrqqqx+di/yN3pF9T9nVfvra7LNqfbElWABcBNw26Fk1JcgJwPnAzQFX9aDEHfudC4L8XY+BPswRYmmQJcByL9z1AbwXuq6r/q6r9wH8C75urnY9c6MNPl0MeBPYC91TVVwZd0yz8LfAnwE8GXcgcKOALSR7oPmpjsfpl4LvAP3fLbjclmZ8PSlk4a4HbBl1Ev6rqKeCvgCeBp4Hnq+oLg62qbw8B5yd5U5LjgHfz6je1zspIhn5VvVxVZzH1jt9zul+XFp0k7wH2VtUDg65ljpxXVW9n6lNWr05y/qAL6tMS4O3AjVX1NuBFYMNgS+pftzz1XuBfBl1Lv5KcyNSHN54G/AKwLMnvDraq/lTVo8BfAvcwtbTzDWD/XO1/JEP/Fd2v3JPAmgGX0q/zgPd2a+FbgN9I8qnBltS/qtrTXe8F7mBqzXIx2g3snvYb5O1M/RBYrN4FfK2qnhl0IbPwm8ATVfXdqvox8Fng1wdcU9+q6uaqentVnQ88C8zJej6MYOgneXOSN3S3lzL1YvjmYKvqT1VdU1UrqmolU79+/3tVLcqjlyTLkrz+ldvAO5j6NXbRqar/Ab6T5C3d0IUs7o8Hv5RFvLTTeRI4N8lx3ckcFwKPDrimviU5ubv+ReC3mcPnZ8H/56wFcAqwuTsb4TXA1qpa1Kc6jogx4I6p70eWAJ+uqs8PtqRZ+UPg1m5p5FvA7w24nr50a8a/BXxo0LXMRlV9JcntwNeYWgr5Oov74xg+k+RNwI+Bq6vqubna8cidsilJOrSRW96RJB2aoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5Ia8v9wPc08ifqwpQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df['quality'].hist(bins=20);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Nut1rmYLzf-p"
   },
   "source": [
    "### Imbalanced data (TODO)\n",
    "You can see from the plot above that the wine quality dataset is imbalanced. \n",
    "- Since there are very few observations with quality equal to 3, 4, 8 and 9, you can drop these observations from your dataset. \n",
    "- You can do this by removing data belonging to all classes except those > 4 and < 8."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "doH9_-gnf3sz",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "d9ba9fc3a3ca02ccc567be33652b80fe",
     "grade": false,
     "grade_id": "cell-6a3e9db696f6827b",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "# get data with wine quality greater than 4 and less than 8\n",
    "df = df[(df['quality'] > 4) & (df['quality'] < 8)]\n",
    "\n",
    "# reset index and drop the old one\n",
    "df = df.reset_index(drop=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "281e1d86a4803560ed5892cd7eda4c01",
     "grade": true,
     "grade_id": "cell-aed3da719d4682c7",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_df_drop(df)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "xNR1iAlMRPXO"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "9.4\n",
      "10.9\n"
     ]
    }
   ],
   "source": [
    "print(df.alcohol[0])\n",
    "print(df.alcohol[100])\n",
    "\n",
    "# EXPECTED OUTPUT\n",
    "# 9.4\n",
    "# 10.9"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "cwhuRpnVRTzG"
   },
   "source": [
    "You can plot again to see the new range of data and quality"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "857ygzZiLgGg"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD4CAYAAAAAczaOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAQaUlEQVR4nO3df6zddX3H8edrRQkW+ZXOO9IywaQx48dE2zDUzN2GRapuK/vDpIQIZCydBpOZkEWYyTRZmuAfbAk4yDpxQGQ2zB8rEdlGGDdmU8TiwPJDtEonpUinIFBiMLD3/jjfbsfLae85p/ec2/p5PpKT8z2f7/fz/b6/Xz687rmfe863qSokSW34laUuQJI0PYa+JDXE0Jekhhj6ktQQQ1+SGnLUUhewkBUrVtSpp546Vt8XX3yR5cuXL25Bi8C6RmNdo7Gu0fyy1nX//ff/uKp+9VUrquqwfqxZs6bGdc8994zdd5KsazTWNRrrGs0va13A9hqQqU7vSFJDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQw772zBIh6sdTz7HpVfeMVbfXVe/b5GrkYbjO31JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDFgz9JKckuSfJo0keTvKnXftJSe5K8r3u+cS+Plcl2ZnksSTn97WvSbKjW3dtkkzmtCRJgwzzTv9l4Iqq+g3gXODyJKcDVwJ3V9Vq4O7uNd26jcAZwHrg+iTLun3dAGwCVneP9Yt4LpKkBSwY+lX1VFV9q1t+AXgUWAlsAG7uNrsZuKBb3gBsraqXqupxYCdwTpKTgeOq6utVVcAtfX0kSVMw0px+klOBtwLfAGaq6ino/WAA3tBtthJ4oq/b7q5tZbc8v12SNCVHDbthkmOBLwAfqarnDzIdP2hFHaR90LE20ZsGYmZmhrm5uWHL/AX79u0bu+8kWddoDte6Zo6BK856eay+kzyfw/V6WddoJlXXUKGf5DX0Av/Wqvpi1/x0kpOr6qlu6mZv174bOKWv+ypgT9e+akD7q1TVFmALwNq1a2t2dna4s5lnbm6OcftOknWN5nCt67pbt3HNjqHfN/2CXRfNLm4xfQ7X62Vdo5lUXcN8eifAjcCjVfVXfatuBy7pli8BtvW1b0xydJLT6P3B9r5uCuiFJOd2+7y4r48kaQqGeZvyTuADwI4kD3Rtfw5cDdyW5DLgh8D7Aarq4SS3AY/Q++TP5VX1StfvQ8BNwDHAnd1DkjQlC4Z+Vf07g+fjAc47QJ/NwOYB7duBM0cpUJK0ePxGriQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDVkvH/V+Qix48nnuPTKO8bqu+vq9y1yNZK09HynL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIQuGfpLPJNmb5KG+tk8keTLJA93jvX3rrkqyM8ljSc7va1+TZEe37tokWfzTkSQdzDDv9G8C1g9o/+uqOrt7fAUgyenARuCMrs/1SZZ1298AbAJWd49B+5QkTdCCoV9VXwWeGXJ/G4CtVfVSVT0O7ATOSXIycFxVfb2qCrgFuGDcoiVJ40kvgxfYKDkV+HJVndm9/gRwKfA8sB24oqqeTfIp4N6q+my33Y3AncAu4Oqq+t2u/beBj1bV7x3geJvo/VbAzMzMmq1bt451cnufeY6nfzZWV85aefx4HYewb98+jj322Intf1zWNRrH12isazSHWte6devur6q189uPGnN/NwB/CVT3fA3wR8Cgefo6SPtAVbUF2AKwdu3amp2dHavI627dxjU7xjvFXReNd8xhzM3NMe45TZJ1jcbxNRrrGs2k6hrr0ztV9XRVvVJV/wP8HXBOt2o3cErfpquAPV37qgHtkqQpGiv0uzn6/f4Q2P/JntuBjUmOTnIavT/Y3ldVTwEvJDm3+9TOxcC2Q6hbkjSGBX83TfI5YBZYkWQ38HFgNsnZ9KZodgF/AlBVDye5DXgEeBm4vKpe6Xb1IXqfBDqG3jz/nYt5IpKkhS0Y+lV14YDmGw+y/WZg84D27cCZI1UnSVpUfiNXkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqyIKhn+QzSfYmeaiv7aQkdyX5Xvd8Yt+6q5LsTPJYkvP72tck2dGtuzZJFv90JEkHM8w7/ZuA9fPargTurqrVwN3da5KcDmwEzuj6XJ9kWdfnBmATsLp7zN+nJGnCFgz9qvoq8My85g3Azd3yzcAFfe1bq+qlqnoc2Amck+Rk4Liq+npVFXBLXx9J0pSMO6c/U1VPAXTPb+jaVwJP9G23u2tb2S3Pb5ckTdFRi7y/QfP0dZD2wTtJNtGbCmJmZoa5ubmxipk5Bq446+Wx+o57zGHs27dvovsfl3WNxvE1GusazaTqGjf0n05yclU91U3d7O3adwOn9G23CtjTta8a0D5QVW0BtgCsXbu2Zmdnxyryulu3cc2O8U5x10XjHXMYc3NzjHtOk2Rdo3F8jca6RjOpusad3rkduKRbvgTY1te+McnRSU6j9wfb+7opoBeSnNt9aufivj6SpClZ8G1Kks8Bs8CKJLuBjwNXA7cluQz4IfB+gKp6OMltwCPAy8DlVfVKt6sP0fsk0DHAnd1DkjRFC4Z+VV14gFXnHWD7zcDmAe3bgTNHqk6StKj8Rq4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ1Z7HvvSJI6p155x9h9b1q/fBEr+X++05ekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDin0k+xKsiPJA0m2d20nJbkryfe65xP7tr8qyc4kjyU5/1CLlySNZjHe6a+rqrOram33+krg7qpaDdzdvSbJ6cBG4AxgPXB9kmWLcHxJ0pAmMb2zAbi5W74ZuKCvfWtVvVRVjwM7gXMmcHxJ0gGkqsbvnDwOPAsU8LdVtSXJT6vqhL5tnq2qE5N8Cri3qj7btd8I3FlVnx+w303AJoCZmZk1W7duHau+vc88x9M/G6srZ608fryOQ9i3bx/HHnvsxPY/LusajeNrNC3WtePJ58bue9rxyw6prnXr1t3fNwPzf44ae48976yqPUneANyV5DsH2TYD2gb+xKmqLcAWgLVr19bs7OxYxV136zau2THeKe66aLxjDmNubo5xz2mSrGs0jq/RtFjXpVfeMXbfm9Yvn0hdhzS9U1V7uue9wJfoTdc8neRkgO55b7f5buCUvu6rgD2HcnxJ0mjGDv0ky5O8fv8y8G7gIeB24JJus0uAbd3y7cDGJEcnOQ1YDdw37vElSaM7lOmdGeBLSfbv5x+q6p+TfBO4LcllwA+B9wNU1cNJbgMeAV4GLq+qVw6peknSSMYO/ar6AfCWAe0/Ac47QJ/NwOZxjylJOjR+I1eSGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5IaYuhLUkMMfUlqiKEvSQ0x9CWpIYa+JDXE0Jekhhj6ktQQQ1+SGmLoS1JDDH1JaoihL0kNMfQlqSGGviQ1xNCXpIYY+pLUEENfkhpi6EtSQwx9SWqIoS9JDZl66CdZn+SxJDuTXDnt40tSy6Ya+kmWAX8DvAc4HbgwyenTrEGSWjbtd/rnADur6gdV9XNgK7BhyjVIUrOOmvLxVgJP9L3eDfzW/I2SbAI2dS/3JXlszOOtAH48Tsd8cswjDmfsuibMukbj+BqNdY1g3ScPua43DmqcduhnQFu9qqFqC7DlkA+WbK+qtYe6n8VmXaOxrtFY12haq2va0zu7gVP6Xq8C9ky5Bklq1rRD/5vA6iSnJXktsBG4fco1SFKzpjq9U1UvJ/kw8C/AMuAzVfXwBA95yFNEE2Jdo7Gu0VjXaJqqK1WvmlKXJP2S8hu5ktQQQ1+SGnLEhn6SXUl2JHkgyfYB65Pk2u52D99O8ra+dRO7FcQQdV3U1fPtJF9L8pZh+064rtkkz3XrH0jyF33rlvJ6/VlfTQ8leSXJScP0PcS6Tkjy+STfSfJokrfPW79U42uhupZqfC1U11KNr4Xqmvr4SvLmvmM+kOT5JB+Zt83kxldVHZEPYBew4iDr3wvcSe+7AecC3+jalwHfB94EvBZ4EDh9inW9AzixW37P/rqG6TvhumaBLw9oX9LrNW/b3wf+bUrX62bgj7vl1wInHCbja6G6lmp8LVTXUo2vg9a1VONr3vn/CHjjtMbXEftOfwgbgFuq517ghCQns8S3gqiqr1XVs93Le+l9V+FwdjjdOuNC4HOTPkiS44B3ATcCVNXPq+qn8zab+vgapq6lGF9DXq8DWdLrNc9Uxtc85wHfr6r/mtc+sfF1JId+Af+a5P70btsw36BbPqw8SPu06up3Gb2f5uP0nURdb0/yYJI7k5zRtR0W1yvJ64D1wBdG7TuGNwH/Dfx9kv9M8ukky+dtsxTja5i6+k1rfA1b17TH19DXa8rjq99GBv+gmdj4OpJD/51V9TZ6v8JenuRd89Yf6JYPQ90KYoJ19YpL1tH7n/Kjo/adUF3fovcr5luA64B/2l/qgH1N/XrR+9X7P6rqmTH6juoo4G3ADVX1VuBFYP7c6VKMr2Hq6hU33fE1TF1LMb6Gvl5Md3wBkN4XVP8A+MdBqwe0Lcr4OmJDv6r2dM97gS/R+7Wn34Fu+TDRW0EMURdJfhP4NLChqn4ySt9J1VVVz1fVvm75K8BrkqzgMLhenVe9I5rg9doN7K6qb3SvP08vPOZvM+3xNUxdSzG+FqxricbXUNerM83xtd97gG9V1dMD1k1sfB2RoZ9keZLX718G3g08NG+z24GLu7+Cnws8V1VPMcFbQQxTV5JfB74IfKCqvjviOU2yrl9Lkm75HHpj4ycs8fXq1h0P/A6wbdS+46iqHwFPJHlz13Qe8Mi8zaY+voapaynG15B1TX18Dfnfcerjq8/B/oYwufG1WH+FnuaD3lzdg93jYeBjXfsHgQ92y6H3D7Z8H9gBrO3r/17gu926j025rk8DzwIPdI/tB+s7xbo+3K17kN4fAN9xOFyv7vWlwNZh+i5ibWcD24Fv05uKOHGpx9eQdU19fA1Z19TH1zB1LeH4eh29H3rH97VNZXx5GwZJasgROb0jSRqPoS9JDTH0Jakhhr4kNcTQl6SGGPqS1BBDX5Ia8r889HO6f1QuqQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df['quality'].hist(bins=20);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "n3k0vqSsp84t"
   },
   "source": [
    "### Train Test Split (TODO)\n",
    "\n",
    "Next, you can split the datasets into training, test and validation datasets.\n",
    "- The data frame should be split 80:20 into `train` and `test` sets.\n",
    "- The resulting `train` should then be split 80:20 into `train` and `val` sets.\n",
    "- The `train_test_split` parameter `test_size` takes a float value that ranges between 0. and 1, and represents the proportion of the dataset that is allocated to the test set.  The rest of the data is allocated to the training set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "PAVIf2-fgRVY",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "7f5738f4fb51d65adc9a8acbdf2b9970",
     "grade": false,
     "grade_id": "cell-91946cadf745206b",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "# Please do not change the random_state parameter. This is needed for grading.\n",
    "\n",
    "# split df into 80:20 train and test sets\n",
    "train, test = train_test_split(df, test_size=0.2, random_state = 1)\n",
    "                               \n",
    "# split train into 80:20 train and val sets\n",
    "train, val = train_test_split(train, test_size=0.2, random_state = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "editable": false,
    "id": "57h9LcEzRWpk",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "42adbe9e66efac7c7a5f8cd73ac92f22",
     "grade": true,
     "grade_id": "cell-64b8b38cd0b965f6",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_data_sizes(train.size, test.size, val.size)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "RwTNu4KFqG-K"
   },
   "source": [
    "Here's where you can explore the training stats. You can pop the labels 'is_red' and 'quality' from the data as these will be used as the labels\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Y_afyhhHM6WQ"
   },
   "outputs": [],
   "source": [
    "train_stats = train.describe()\n",
    "train_stats.pop('is_red')\n",
    "train_stats.pop('quality')\n",
    "train_stats = train_stats.transpose()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ahvbYm4fNqSt"
   },
   "source": [
    "Explore the training stats!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "n_gAtPjZ0otF"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>fixed acidity</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>7.221616</td>\n",
       "      <td>1.325297</td>\n",
       "      <td>3.80000</td>\n",
       "      <td>6.40000</td>\n",
       "      <td>7.00000</td>\n",
       "      <td>7.7000</td>\n",
       "      <td>15.60000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>volatile acidity</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>0.338929</td>\n",
       "      <td>0.162476</td>\n",
       "      <td>0.08000</td>\n",
       "      <td>0.23000</td>\n",
       "      <td>0.29000</td>\n",
       "      <td>0.4000</td>\n",
       "      <td>1.24000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>citric acid</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>0.321569</td>\n",
       "      <td>0.147970</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.25000</td>\n",
       "      <td>0.31000</td>\n",
       "      <td>0.4000</td>\n",
       "      <td>1.66000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>residual sugar</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>5.155911</td>\n",
       "      <td>4.639632</td>\n",
       "      <td>0.60000</td>\n",
       "      <td>1.80000</td>\n",
       "      <td>2.80000</td>\n",
       "      <td>7.6500</td>\n",
       "      <td>65.80000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>chlorides</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>0.056976</td>\n",
       "      <td>0.036802</td>\n",
       "      <td>0.01200</td>\n",
       "      <td>0.03800</td>\n",
       "      <td>0.04700</td>\n",
       "      <td>0.0660</td>\n",
       "      <td>0.61100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>free sulfur dioxide</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>30.388590</td>\n",
       "      <td>17.236784</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>17.00000</td>\n",
       "      <td>28.00000</td>\n",
       "      <td>41.0000</td>\n",
       "      <td>131.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>total sulfur dioxide</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>115.062282</td>\n",
       "      <td>56.706617</td>\n",
       "      <td>6.00000</td>\n",
       "      <td>75.00000</td>\n",
       "      <td>117.00000</td>\n",
       "      <td>156.0000</td>\n",
       "      <td>344.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>density</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>0.994633</td>\n",
       "      <td>0.003005</td>\n",
       "      <td>0.98711</td>\n",
       "      <td>0.99232</td>\n",
       "      <td>0.99481</td>\n",
       "      <td>0.9968</td>\n",
       "      <td>1.03898</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>pH</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>3.223201</td>\n",
       "      <td>0.161272</td>\n",
       "      <td>2.72000</td>\n",
       "      <td>3.11000</td>\n",
       "      <td>3.21000</td>\n",
       "      <td>3.3300</td>\n",
       "      <td>4.01000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sulphates</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>0.534051</td>\n",
       "      <td>0.149149</td>\n",
       "      <td>0.22000</td>\n",
       "      <td>0.43000</td>\n",
       "      <td>0.51000</td>\n",
       "      <td>0.6000</td>\n",
       "      <td>1.95000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>alcohol</th>\n",
       "      <td>3155.0</td>\n",
       "      <td>10.504466</td>\n",
       "      <td>1.154654</td>\n",
       "      <td>8.50000</td>\n",
       "      <td>9.50000</td>\n",
       "      <td>10.30000</td>\n",
       "      <td>11.3000</td>\n",
       "      <td>14.00000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                       count        mean        std      min       25%  \\\n",
       "fixed acidity         3155.0    7.221616   1.325297  3.80000   6.40000   \n",
       "volatile acidity      3155.0    0.338929   0.162476  0.08000   0.23000   \n",
       "citric acid           3155.0    0.321569   0.147970  0.00000   0.25000   \n",
       "residual sugar        3155.0    5.155911   4.639632  0.60000   1.80000   \n",
       "chlorides             3155.0    0.056976   0.036802  0.01200   0.03800   \n",
       "free sulfur dioxide   3155.0   30.388590  17.236784  1.00000  17.00000   \n",
       "total sulfur dioxide  3155.0  115.062282  56.706617  6.00000  75.00000   \n",
       "density               3155.0    0.994633   0.003005  0.98711   0.99232   \n",
       "pH                    3155.0    3.223201   0.161272  2.72000   3.11000   \n",
       "sulphates             3155.0    0.534051   0.149149  0.22000   0.43000   \n",
       "alcohol               3155.0   10.504466   1.154654  8.50000   9.50000   \n",
       "\n",
       "                            50%       75%        max  \n",
       "fixed acidity           7.00000    7.7000   15.60000  \n",
       "volatile acidity        0.29000    0.4000    1.24000  \n",
       "citric acid             0.31000    0.4000    1.66000  \n",
       "residual sugar          2.80000    7.6500   65.80000  \n",
       "chlorides               0.04700    0.0660    0.61100  \n",
       "free sulfur dioxide    28.00000   41.0000  131.00000  \n",
       "total sulfur dioxide  117.00000  156.0000  344.00000  \n",
       "density                 0.99481    0.9968    1.03898  \n",
       "pH                      3.21000    3.3300    4.01000  \n",
       "sulphates               0.51000    0.6000    1.95000  \n",
       "alcohol                10.30000   11.3000   14.00000  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_stats"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bGPvt9jir_HC"
   },
   "source": [
    "### Get the labels (TODO)\n",
    "\n",
    "The features and labels are currently in the same dataframe.\n",
    "- You will want to store the label columns `is_red` and `quality` separately from the feature columns.  \n",
    "- The following function, `format_output`, gets these two columns from the dataframe (it's given to you).\n",
    "- `format_output` also formats the data into numpy arrays. \n",
    "- Please use the `format_output` and apply it to the `train`, `val` and `test` sets to get dataframes for the labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Z_fs14XQqZVP"
   },
   "outputs": [],
   "source": [
    "def format_output(data):\n",
    "    is_red = data.pop('is_red')\n",
    "    is_red = np.array(is_red)\n",
    "    quality = data.pop('quality')\n",
    "    quality = np.array(quality)\n",
    "    return (quality, is_red)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "8L3ZZe1fQicm",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "7a86809e54895a816434c48dc903f55d",
     "grade": false,
     "grade_id": "cell-5c30fa2c2a354b0f",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "# format the output of the train set\n",
    "train_Y = format_output(train)\n",
    "\n",
    "# format the output of the val set\n",
    "val_Y = format_output(val)\n",
    "    \n",
    "# format the output of the test set\n",
    "test_Y = format_output(test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "359cabbafaed14ec9bbc1e57a7b6f32c",
     "grade": true,
     "grade_id": "cell-4977d8befb80f56b",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_format_output(df, train_Y, val_Y, test_Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that after you get the labels, the `train`, `val` and `test` dataframes no longer contain the label columns, and contain just the feature columns.\n",
    "- This is because you used `.pop` in the `format_output` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fixed acidity</th>\n",
       "      <th>volatile acidity</th>\n",
       "      <th>citric acid</th>\n",
       "      <th>residual sugar</th>\n",
       "      <th>chlorides</th>\n",
       "      <th>free sulfur dioxide</th>\n",
       "      <th>total sulfur dioxide</th>\n",
       "      <th>density</th>\n",
       "      <th>pH</th>\n",
       "      <th>sulphates</th>\n",
       "      <th>alcohol</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>225</th>\n",
       "      <td>7.5</td>\n",
       "      <td>0.65</td>\n",
       "      <td>0.18</td>\n",
       "      <td>7.0</td>\n",
       "      <td>0.088</td>\n",
       "      <td>27.0</td>\n",
       "      <td>94.0</td>\n",
       "      <td>0.99915</td>\n",
       "      <td>3.38</td>\n",
       "      <td>0.77</td>\n",
       "      <td>9.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3557</th>\n",
       "      <td>6.3</td>\n",
       "      <td>0.27</td>\n",
       "      <td>0.29</td>\n",
       "      <td>12.2</td>\n",
       "      <td>0.044</td>\n",
       "      <td>59.0</td>\n",
       "      <td>196.0</td>\n",
       "      <td>0.99782</td>\n",
       "      <td>3.14</td>\n",
       "      <td>0.40</td>\n",
       "      <td>8.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3825</th>\n",
       "      <td>8.8</td>\n",
       "      <td>0.27</td>\n",
       "      <td>0.25</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.024</td>\n",
       "      <td>52.0</td>\n",
       "      <td>99.0</td>\n",
       "      <td>0.99250</td>\n",
       "      <td>2.87</td>\n",
       "      <td>0.49</td>\n",
       "      <td>11.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1740</th>\n",
       "      <td>6.4</td>\n",
       "      <td>0.45</td>\n",
       "      <td>0.07</td>\n",
       "      <td>1.1</td>\n",
       "      <td>0.030</td>\n",
       "      <td>10.0</td>\n",
       "      <td>131.0</td>\n",
       "      <td>0.99050</td>\n",
       "      <td>2.97</td>\n",
       "      <td>0.28</td>\n",
       "      <td>10.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1221</th>\n",
       "      <td>7.2</td>\n",
       "      <td>0.53</td>\n",
       "      <td>0.13</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.058</td>\n",
       "      <td>18.0</td>\n",
       "      <td>22.0</td>\n",
       "      <td>0.99573</td>\n",
       "      <td>3.21</td>\n",
       "      <td>0.68</td>\n",
       "      <td>9.9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      fixed acidity  volatile acidity  citric acid  residual sugar  chlorides  \\\n",
       "225             7.5              0.65         0.18             7.0      0.088   \n",
       "3557            6.3              0.27         0.29            12.2      0.044   \n",
       "3825            8.8              0.27         0.25             5.0      0.024   \n",
       "1740            6.4              0.45         0.07             1.1      0.030   \n",
       "1221            7.2              0.53         0.13             2.0      0.058   \n",
       "\n",
       "      free sulfur dioxide  total sulfur dioxide  density    pH  sulphates  \\\n",
       "225                  27.0                  94.0  0.99915  3.38       0.77   \n",
       "3557                 59.0                 196.0  0.99782  3.14       0.40   \n",
       "3825                 52.0                  99.0  0.99250  2.87       0.49   \n",
       "1740                 10.0                 131.0  0.99050  2.97       0.28   \n",
       "1221                 18.0                  22.0  0.99573  3.21       0.68   \n",
       "\n",
       "      alcohol  \n",
       "225       9.4  \n",
       "3557      8.8  \n",
       "3825     11.4  \n",
       "1740     10.8  \n",
       "1221      9.9  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hEdbrruAsN1D"
   },
   "source": [
    "### Normalize the data (TODO)\n",
    "\n",
    "Next, you can normalize the data, x, using the formula:\n",
    "$$x_{norm} = \\frac{x - \\mu}{\\sigma}$$\n",
    "- The `norm` function is defined for you.\n",
    "- Please apply the `norm` function to normalize the dataframes that contains the feature columns of `train`, `val` and `test` sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WWiZPAHCLjUs"
   },
   "outputs": [],
   "source": [
    "def norm(x):\n",
    "    return (x - train_stats['mean']) / train_stats['std']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "JEaOi2I2Lk69",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "6bc0cdcb563d192f271067aa3373ff32",
     "grade": false,
     "grade_id": "cell-d8416d975c371095",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "# normalize the train set\n",
    "norm_train_X = norm(train)\n",
    "    \n",
    "# normalize the val set\n",
    "norm_val_X = norm(val)\n",
    "    \n",
    "# normalize the test set\n",
    "norm_test_X = norm(test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "4f567db45bf40191601780379cc100b8",
     "grade": true,
     "grade_id": "cell-97fad979d157529b",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_norm(norm_train_X, norm_val_X, norm_test_X, train, val, test)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "hzykDwQhsaPO"
   },
   "source": [
    "## Define the Model (TODO)\n",
    "\n",
    "Define the model using the functional API. The base model will be 2 `Dense` layers of 128 neurons each, and have the `'relu'` activation.\n",
    "- Check out the documentation for [tf.keras.layers.Dense](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "Rhcns3oTFkM6",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "74b031247e569526552bf13a034a1c07",
     "grade": false,
     "grade_id": "cell-73fceedad1fe351c",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "def base_model(inputs):\n",
    "    \n",
    "    # connect a Dense layer with 128 neurons and a relu activation\n",
    "    x = Dense(128, activation = 'relu')(inputs)\n",
    "    \n",
    "    # connect another Dense layer with 128 neurons and a relu activation\n",
    "    x = Dense(128, activation = 'relu')(x)\n",
    "    return x\n",
    "  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "9255924b3def80f679616e4c851a43e1",
     "grade": true,
     "grade_id": "cell-54f742a133353d75",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_base_model(base_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "xem_fcVws6Kz"
   },
   "source": [
    "# Define output layers of the model (TODO)\n",
    "\n",
    "You will add output layers to the base model. \n",
    "- The model will need two outputs.\n",
    "\n",
    "One output layer will predict wine quality, which is a numeric value.\n",
    "- Define a `Dense` layer with 1 neuron.\n",
    "- Since this is a regression output, the activation can be left as its default value `None`.\n",
    "\n",
    "The other output layer will predict the wine type, which is either red `1` or not red `0` (white).\n",
    "- Define a `Dense` layer with 1 neuron.\n",
    "- Since there are two possible categories, you can use a sigmoid activation for binary classification.\n",
    "\n",
    "Define the `Model`\n",
    "- Define the `Model` object, and set the following parameters:\n",
    "  - `inputs`: pass in the inputs to the model as a list.\n",
    "  - `outputs`: pass in a list of the outputs that you just defined: wine quality, then wine type.\n",
    "  - **Note**: please list the wine quality before wine type in the outputs, as this will affect the calculated loss if you choose the other order."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "n5UGF8PMVLPt",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "76d35b90d20cdcbb22986cd8211057de",
     "grade": false,
     "grade_id": "cell-19e285f482f021fb",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "def final_model(inputs):\n",
    "    \n",
    "    # get the base model\n",
    "    x = base_model(inputs)\n",
    "\n",
    "    # connect the output Dense layer for regression\n",
    "    wine_quality = Dense(units='1', name='wine_quality')(x)\n",
    "\n",
    "    # connect the output Dense layer for classification. this will use a sigmoid activation.\n",
    "    wine_type = Dense(units='1', activation='sigmoid', name='wine_type')(x)\n",
    "\n",
    "    # define the model using the input and output layers\n",
    "    model = Model(inputs=inputs, outputs=[wine_quality, wine_type])\n",
    "\n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "89cbf89d8ab5e2e59ecf7f63f517520a",
     "grade": true,
     "grade_id": "cell-40d050f855c817d1",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_final_model(final_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "5R0BMTsltZyu"
   },
   "source": [
    "## Compiling the Model\n",
    "\n",
    "Next, compile the model. When setting the loss parameter of `model.compile`, you're setting the loss for each of the two outputs (wine quality and wine type).\n",
    "\n",
    "To set more than one loss, use a dictionary of key-value pairs.\n",
    "- You can look at the docs for the losses [here](https://www.tensorflow.org/api_docs/python/tf/keras/losses#functions).\n",
    "    - **Note**: For the desired spelling, please look at the \"Functions\" section of the documentation and not the \"classes\" section on that same page.\n",
    "- wine_type: Since you will be performing binary classification on wine type, you should use the binary crossentropy loss function for it.  Please pass this in as a string.  \n",
    "  - **Hint**, this should be all lowercase.  In the documentation, you'll see this under the \"Functions\" section, not the \"Classes\" section.\n",
    "- wine_quality: since this is a regression output, use the mean squared error.  Please pass it in as a string, all lowercase.\n",
    "  - **Hint**: You may notice that there are two aliases for mean squared error.  Please use the shorter name.\n",
    "\n",
    "\n",
    "You will also set the metric for each of the two outputs.  Again, to set metrics for two or more outputs, use a dictionary with key value pairs.\n",
    "- The metrics documentation is linked [here](https://www.tensorflow.org/api_docs/python/tf/keras/metrics).\n",
    "- For the wine type, please set it to accuracy as a string, all lowercase.\n",
    "- For wine quality, please use the root mean squared error.  Instead of a string, you'll set it to an instance of the class [RootMeanSquaredError](https://www.tensorflow.org/api_docs/python/tf/keras/metrics/RootMeanSquaredError), which belongs to the tf.keras.metrics module.\n",
    "\n",
    "**Note**: If you see the error message \n",
    ">Exception: wine quality loss function is incorrect.\n",
    "\n",
    "- Please also check your other losses and metrics, as the error may be caused by the other three key-value pairs and not the wine quality loss."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "LK11duUbUjmh",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "22f45067ca69eb2ccadb43874dbcc27b",
     "grade": false,
     "grade_id": "cell-81afdc4dcca51d5e",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "inputs = tf.keras.layers.Input(shape=(11,))\n",
    "rms = tf.keras.optimizers.RMSprop(lr=0.0001)\n",
    "model = final_model(inputs)\n",
    "\n",
    "model.compile(optimizer=rms, \n",
    "              loss = {'wine_type' : 'binary_crossentropy',\n",
    "                      'wine_quality' : 'mse'\n",
    "                     },\n",
    "              metrics = {'wine_type' : 'accuracy',\n",
    "                         'wine_quality': tf.keras.metrics.RootMeanSquaredError()\n",
    "                       }\n",
    "             )\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "88e02238ea5e456ff65e835cc8158054",
     "grade": true,
     "grade_id": "cell-2eeeba02391c4632",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_model_compile(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "90MpAMpWuKm-"
   },
   "source": [
    "## Training the Model (TODO)\n",
    "\n",
    "Fit the model to the training inputs and outputs. \n",
    "- Check the documentation for [model.fit](https://www.tensorflow.org/api_docs/python/tf/keras/Model#fit).\n",
    "- Remember to use the normalized training set as inputs. \n",
    "- For the validation data, please use the normalized validation set.\n",
    "\n",
    "**Important: Please do not increase the number of epochs below. This is to avoid the grader from timing out. You can increase it once you have submitted your work.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "deletable": false,
    "id": "_eiZkle4XwiY",
    "nbgrader": {
     "cell_type": "code",
     "checksum": "d1a4565296017a0611c6f2de675f96cf",
     "grade": false,
     "grade_id": "cell-0bb56262896f6680",
     "locked": false,
     "schema_version": 3,
     "solution": true,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 3155 samples, validate on 789 samples\n",
      "Epoch 1/40\n",
      "3155/3155 [==============================] - 1s 354us/sample - loss: 26.2828 - wine_quality_loss: 25.6000 - wine_type_loss: 0.6549 - wine_quality_root_mean_squared_error: 5.0624 - wine_type_accuracy: 0.7208 - val_loss: 18.2175 - val_wine_quality_loss: 17.6046 - val_wine_type_loss: 0.6229 - val_wine_quality_root_mean_squared_error: 4.1946 - val_wine_type_accuracy: 0.8112\n",
      "Epoch 2/40\n",
      "3155/3155 [==============================] - 0s 96us/sample - loss: 12.1116 - wine_quality_loss: 11.5054 - wine_type_loss: 0.5872 - wine_quality_root_mean_squared_error: 3.3947 - wine_type_accuracy: 0.7873 - val_loss: 6.9483 - val_wine_quality_loss: 6.4233 - val_wine_type_loss: 0.5588 - val_wine_quality_root_mean_squared_error: 2.5278 - val_wine_type_accuracy: 0.7376\n",
      "Epoch 3/40\n",
      "3155/3155 [==============================] - 0s 94us/sample - loss: 4.5043 - wine_quality_loss: 3.9894 - wine_type_loss: 0.5134 - wine_quality_root_mean_squared_error: 1.9977 - wine_type_accuracy: 0.7471 - val_loss: 2.9075 - val_wine_quality_loss: 2.4770 - val_wine_type_loss: 0.4749 - val_wine_quality_root_mean_squared_error: 1.5598 - val_wine_type_accuracy: 0.7402\n",
      "Epoch 4/40\n",
      "3155/3155 [==============================] - 0s 95us/sample - loss: 2.7102 - wine_quality_loss: 2.3530 - wine_type_loss: 0.4188 - wine_quality_root_mean_squared_error: 1.5136 - wine_type_accuracy: 0.8051 - val_loss: 2.3664 - val_wine_quality_loss: 2.0134 - val_wine_type_loss: 0.3805 - val_wine_quality_root_mean_squared_error: 1.4093 - val_wine_type_accuracy: 0.8606\n",
      "Epoch 5/40\n",
      "3155/3155 [==============================] - 0s 94us/sample - loss: 2.2823 - wine_quality_loss: 1.9437 - wine_type_loss: 0.3367 - wine_quality_root_mean_squared_error: 1.3947 - wine_type_accuracy: 0.9081 - val_loss: 2.0602 - val_wine_quality_loss: 1.7718 - val_wine_type_loss: 0.3078 - val_wine_quality_root_mean_squared_error: 1.3239 - val_wine_type_accuracy: 0.9366\n",
      "Epoch 6/40\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 2.0017 - wine_quality_loss: 1.7273 - wine_type_loss: 0.2701 - wine_quality_root_mean_squared_error: 1.3159 - wine_type_accuracy: 0.9556 - val_loss: 1.8217 - val_wine_quality_loss: 1.5880 - val_wine_type_loss: 0.2474 - val_wine_quality_root_mean_squared_error: 1.2548 - val_wine_type_accuracy: 0.9708\n",
      "Epoch 7/40\n",
      "3155/3155 [==============================] - 0s 96us/sample - loss: 1.7862 - wine_quality_loss: 1.5683 - wine_type_loss: 0.2166 - wine_quality_root_mean_squared_error: 1.2528 - wine_type_accuracy: 0.9724 - val_loss: 1.6330 - val_wine_quality_loss: 1.4441 - val_wine_type_loss: 0.1987 - val_wine_quality_root_mean_squared_error: 1.1977 - val_wine_type_accuracy: 0.9848\n",
      "Epoch 8/40\n",
      "3155/3155 [==============================] - 0s 76us/sample - loss: 1.6205 - wine_quality_loss: 1.4435 - wine_type_loss: 0.1752 - wine_quality_root_mean_squared_error: 1.2022 - wine_type_accuracy: 0.9803 - val_loss: 1.4896 - val_wine_quality_loss: 1.3349 - val_wine_type_loss: 0.1610 - val_wine_quality_root_mean_squared_error: 1.1527 - val_wine_type_accuracy: 0.9911\n",
      "Epoch 9/40\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 1.4739 - wine_quality_loss: 1.3315 - wine_type_loss: 0.1434 - wine_quality_root_mean_squared_error: 1.1534 - wine_type_accuracy: 0.9854 - val_loss: 1.3753 - val_wine_quality_loss: 1.2475 - val_wine_type_loss: 0.1324 - val_wine_quality_root_mean_squared_error: 1.1149 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 10/40\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 1.3644 - wine_quality_loss: 1.2444 - wine_type_loss: 0.1202 - wine_quality_root_mean_squared_error: 1.1154 - wine_type_accuracy: 0.9876 - val_loss: 1.2736 - val_wine_quality_loss: 1.1657 - val_wine_type_loss: 0.1104 - val_wine_quality_root_mean_squared_error: 1.0785 - val_wine_type_accuracy: 0.9911\n",
      "Epoch 11/40\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 1.2601 - wine_quality_loss: 1.1568 - wine_type_loss: 0.1018 - wine_quality_root_mean_squared_error: 1.0763 - wine_type_accuracy: 0.9899 - val_loss: 1.1753 - val_wine_quality_loss: 1.0833 - val_wine_type_loss: 0.0933 - val_wine_quality_root_mean_squared_error: 1.0401 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 12/40\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 1.1699 - wine_quality_loss: 1.0829 - wine_type_loss: 0.0879 - wine_quality_root_mean_squared_error: 1.0402 - wine_type_accuracy: 0.9908 - val_loss: 1.0866 - val_wine_quality_loss: 1.0068 - val_wine_type_loss: 0.0807 - val_wine_quality_root_mean_squared_error: 1.0029 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 13/40\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 1.0916 - wine_quality_loss: 1.0125 - wine_type_loss: 0.0774 - wine_quality_root_mean_squared_error: 1.0071 - wine_type_accuracy: 0.9905 - val_loss: 1.0130 - val_wine_quality_loss: 0.9420 - val_wine_type_loss: 0.0713 - val_wine_quality_root_mean_squared_error: 0.9703 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 14/40\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 1.0228 - wine_quality_loss: 0.9514 - wine_type_loss: 0.0693 - wine_quality_root_mean_squared_error: 0.9764 - wine_type_accuracy: 0.9905 - val_loss: 0.9391 - val_wine_quality_loss: 0.8752 - val_wine_type_loss: 0.0641 - val_wine_quality_root_mean_squared_error: 0.9353 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 15/40\n",
      "3155/3155 [==============================] - 0s 76us/sample - loss: 0.9589 - wine_quality_loss: 0.8947 - wine_type_loss: 0.0632 - wine_quality_root_mean_squared_error: 0.9464 - wine_type_accuracy: 0.9908 - val_loss: 0.8992 - val_wine_quality_loss: 0.8406 - val_wine_type_loss: 0.0582 - val_wine_quality_root_mean_squared_error: 0.9169 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 16/40\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.9028 - wine_quality_loss: 0.8458 - wine_type_loss: 0.0582 - wine_quality_root_mean_squared_error: 0.9189 - wine_type_accuracy: 0.9914 - val_loss: 0.8378 - val_wine_quality_loss: 0.7833 - val_wine_type_loss: 0.0539 - val_wine_quality_root_mean_squared_error: 0.8852 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 17/40\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.8464 - wine_quality_loss: 0.7914 - wine_type_loss: 0.0543 - wine_quality_root_mean_squared_error: 0.8900 - wine_type_accuracy: 0.9914 - val_loss: 0.7828 - val_wine_quality_loss: 0.7318 - val_wine_type_loss: 0.0503 - val_wine_quality_root_mean_squared_error: 0.8557 - val_wine_type_accuracy: 0.9924\n",
      "Epoch 18/40\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.7999 - wine_quality_loss: 0.7480 - wine_type_loss: 0.0519 - wine_quality_root_mean_squared_error: 0.8654 - wine_type_accuracy: 0.9921 - val_loss: 0.7453 - val_wine_quality_loss: 0.6966 - val_wine_type_loss: 0.0474 - val_wine_quality_root_mean_squared_error: 0.8352 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 19/40\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.7542 - wine_quality_loss: 0.7058 - wine_type_loss: 0.0483 - wine_quality_root_mean_squared_error: 0.8401 - wine_type_accuracy: 0.9921 - val_loss: 0.7067 - val_wine_quality_loss: 0.6604 - val_wine_type_loss: 0.0452 - val_wine_quality_root_mean_squared_error: 0.8132 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 20/40\n",
      "3155/3155 [==============================] - 0s 92us/sample - loss: 0.7141 - wine_quality_loss: 0.6676 - wine_type_loss: 0.0462 - wine_quality_root_mean_squared_error: 0.8172 - wine_type_accuracy: 0.9924 - val_loss: 0.6668 - val_wine_quality_loss: 0.6220 - val_wine_type_loss: 0.0434 - val_wine_quality_root_mean_squared_error: 0.7894 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 21/40\n",
      "3155/3155 [==============================] - 0s 93us/sample - loss: 0.6793 - wine_quality_loss: 0.6345 - wine_type_loss: 0.0444 - wine_quality_root_mean_squared_error: 0.7968 - wine_type_accuracy: 0.9924 - val_loss: 0.6333 - val_wine_quality_loss: 0.5901 - val_wine_type_loss: 0.0418 - val_wine_quality_root_mean_squared_error: 0.7688 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 22/40\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.6449 - wine_quality_loss: 0.6013 - wine_type_loss: 0.0429 - wine_quality_root_mean_squared_error: 0.7760 - wine_type_accuracy: 0.9924 - val_loss: 0.5986 - val_wine_quality_loss: 0.5567 - val_wine_type_loss: 0.0406 - val_wine_quality_root_mean_squared_error: 0.7468 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 23/40\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.6165 - wine_quality_loss: 0.5741 - wine_type_loss: 0.0414 - wine_quality_root_mean_squared_error: 0.7583 - wine_type_accuracy: 0.9927 - val_loss: 0.5753 - val_wine_quality_loss: 0.5344 - val_wine_type_loss: 0.0395 - val_wine_quality_root_mean_squared_error: 0.7317 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 24/40\n",
      "3155/3155 [==============================] - 0s 76us/sample - loss: 0.5879 - wine_quality_loss: 0.5476 - wine_type_loss: 0.0402 - wine_quality_root_mean_squared_error: 0.7400 - wine_type_accuracy: 0.9927 - val_loss: 0.5587 - val_wine_quality_loss: 0.5186 - val_wine_type_loss: 0.0385 - val_wine_quality_root_mean_squared_error: 0.7210 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 25/40\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.5627 - wine_quality_loss: 0.5238 - wine_type_loss: 0.0391 - wine_quality_root_mean_squared_error: 0.7235 - wine_type_accuracy: 0.9927 - val_loss: 0.5238 - val_wine_quality_loss: 0.4844 - val_wine_type_loss: 0.0378 - val_wine_quality_root_mean_squared_error: 0.6969 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 26/40\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.5404 - wine_quality_loss: 0.5009 - wine_type_loss: 0.0382 - wine_quality_root_mean_squared_error: 0.7086 - wine_type_accuracy: 0.9927 - val_loss: 0.5098 - val_wine_quality_loss: 0.4712 - val_wine_type_loss: 0.0371 - val_wine_quality_root_mean_squared_error: 0.6873 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 27/40\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.5184 - wine_quality_loss: 0.4806 - wine_type_loss: 0.0374 - wine_quality_root_mean_squared_error: 0.6935 - wine_type_accuracy: 0.9927 - val_loss: 0.4888 - val_wine_quality_loss: 0.4505 - val_wine_type_loss: 0.0365 - val_wine_quality_root_mean_squared_error: 0.6723 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 28/40\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.4994 - wine_quality_loss: 0.4620 - wine_type_loss: 0.0365 - wine_quality_root_mean_squared_error: 0.6803 - wine_type_accuracy: 0.9927 - val_loss: 0.4726 - val_wine_quality_loss: 0.4349 - val_wine_type_loss: 0.0360 - val_wine_quality_root_mean_squared_error: 0.6604 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 29/40\n",
      "3155/3155 [==============================] - 0s 91us/sample - loss: 0.4834 - wine_quality_loss: 0.4470 - wine_type_loss: 0.0359 - wine_quality_root_mean_squared_error: 0.6689 - wine_type_accuracy: 0.9927 - val_loss: 0.4652 - val_wine_quality_loss: 0.4279 - val_wine_type_loss: 0.0356 - val_wine_quality_root_mean_squared_error: 0.6551 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 30/40\n",
      "3155/3155 [==============================] - 0s 76us/sample - loss: 0.4688 - wine_quality_loss: 0.4348 - wine_type_loss: 0.0353 - wine_quality_root_mean_squared_error: 0.6584 - wine_type_accuracy: 0.9930 - val_loss: 0.4587 - val_wine_quality_loss: 0.4219 - val_wine_type_loss: 0.0351 - val_wine_quality_root_mean_squared_error: 0.6505 - val_wine_type_accuracy: 0.9937\n",
      "Epoch 31/40\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.4559 - wine_quality_loss: 0.4210 - wine_type_loss: 0.0348 - wine_quality_root_mean_squared_error: 0.6489 - wine_type_accuracy: 0.9930 - val_loss: 0.4392 - val_wine_quality_loss: 0.4029 - val_wine_type_loss: 0.0349 - val_wine_quality_root_mean_squared_error: 0.6356 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 32/40\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.4434 - wine_quality_loss: 0.4089 - wine_type_loss: 0.0341 - wine_quality_root_mean_squared_error: 0.6397 - wine_type_accuracy: 0.9933 - val_loss: 0.4243 - val_wine_quality_loss: 0.3882 - val_wine_type_loss: 0.0346 - val_wine_quality_root_mean_squared_error: 0.6240 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 33/40\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.4315 - wine_quality_loss: 0.3977 - wine_type_loss: 0.0337 - wine_quality_root_mean_squared_error: 0.6306 - wine_type_accuracy: 0.9930 - val_loss: 0.4163 - val_wine_quality_loss: 0.3806 - val_wine_type_loss: 0.0342 - val_wine_quality_root_mean_squared_error: 0.6178 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 34/40\n",
      "3155/3155 [==============================] - 0s 75us/sample - loss: 0.4224 - wine_quality_loss: 0.3894 - wine_type_loss: 0.0332 - wine_quality_root_mean_squared_error: 0.6238 - wine_type_accuracy: 0.9930 - val_loss: 0.4084 - val_wine_quality_loss: 0.3731 - val_wine_type_loss: 0.0340 - val_wine_quality_root_mean_squared_error: 0.6116 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 35/40\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.4133 - wine_quality_loss: 0.3804 - wine_type_loss: 0.0328 - wine_quality_root_mean_squared_error: 0.6168 - wine_type_accuracy: 0.9940 - val_loss: 0.3998 - val_wine_quality_loss: 0.3648 - val_wine_type_loss: 0.0337 - val_wine_quality_root_mean_squared_error: 0.6048 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 36/40\n",
      "3155/3155 [==============================] - 0s 88us/sample - loss: 0.4060 - wine_quality_loss: 0.3741 - wine_type_loss: 0.0323 - wine_quality_root_mean_squared_error: 0.6113 - wine_type_accuracy: 0.9937 - val_loss: 0.3924 - val_wine_quality_loss: 0.3576 - val_wine_type_loss: 0.0335 - val_wine_quality_root_mean_squared_error: 0.5988 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 37/40\n",
      "3155/3155 [==============================] - 0s 76us/sample - loss: 0.3989 - wine_quality_loss: 0.3664 - wine_type_loss: 0.0320 - wine_quality_root_mean_squared_error: 0.6057 - wine_type_accuracy: 0.9937 - val_loss: 0.3895 - val_wine_quality_loss: 0.3547 - val_wine_type_loss: 0.0333 - val_wine_quality_root_mean_squared_error: 0.5965 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 38/40\n",
      "3155/3155 [==============================] - 0s 90us/sample - loss: 0.3938 - wine_quality_loss: 0.3626 - wine_type_loss: 0.0326 - wine_quality_root_mean_squared_error: 0.6017 - wine_type_accuracy: 0.9943 - val_loss: 0.3908 - val_wine_quality_loss: 0.3567 - val_wine_type_loss: 0.0330 - val_wine_quality_root_mean_squared_error: 0.5978 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 39/40\n",
      "3155/3155 [==============================] - 0s 89us/sample - loss: 0.3885 - wine_quality_loss: 0.3575 - wine_type_loss: 0.0312 - wine_quality_root_mean_squared_error: 0.5977 - wine_type_accuracy: 0.9943 - val_loss: 0.3846 - val_wine_quality_loss: 0.3502 - val_wine_type_loss: 0.0330 - val_wine_quality_root_mean_squared_error: 0.5927 - val_wine_type_accuracy: 0.9949\n",
      "Epoch 40/40\n",
      "3155/3155 [==============================] - 0s 74us/sample - loss: 0.3825 - wine_quality_loss: 0.3510 - wine_type_loss: 0.0309 - wine_quality_root_mean_squared_error: 0.5929 - wine_type_accuracy: 0.9943 - val_loss: 0.3815 - val_wine_quality_loss: 0.3477 - val_wine_type_loss: 0.0326 - val_wine_quality_root_mean_squared_error: 0.5903 - val_wine_type_accuracy: 0.9949\n"
     ]
    }
   ],
   "source": [
    "# Please uncomment all lines in this cell and replace those marked with `# YOUR CODE HERE`.\n",
    "# You can select all lines in this code cell with Ctrl+A (Windows/Linux) or Cmd+A (Mac), then press Ctrl+/ (Windows/Linux) or Cmd+/ (Mac) to uncomment.\n",
    "\n",
    "\n",
    "\n",
    "history = model.fit(norm_train_X, train_Y,\n",
    "                    epochs = 40, validation_data=(norm_val_X, val_Y))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "deletable": false,
    "editable": false,
    "nbgrader": {
     "cell_type": "code",
     "checksum": "fadad8896eda9c8c2115970724b15508",
     "grade": true,
     "grade_id": "cell-eb4d5b41bef8f0ab",
     "locked": true,
     "points": 1,
     "schema_version": 3,
     "solution": false,
     "task": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[92m All public tests passed\n"
     ]
    }
   ],
   "source": [
    "utils.test_history(history)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "CubF2J2gSf6q"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "789/789 [==============================] - 0s 21us/sample - loss: 0.3815 - wine_quality_loss: 0.3477 - wine_type_loss: 0.0326 - wine_quality_root_mean_squared_error: 0.5903 - wine_type_accuracy: 0.9949\n",
      "\n",
      "loss: 0.381475464812703\n",
      "wine_quality_loss: 0.3477051556110382\n",
      "wine_type_loss: 0.03259332478046417\n",
      "wine_quality_rmse: 0.590336799621582\n",
      "wine_type_accuracy: 0.9949302673339844\n"
     ]
    }
   ],
   "source": [
    "# Gather the training metrics\n",
    "loss, wine_quality_loss, wine_type_loss, wine_quality_rmse, wine_type_accuracy = model.evaluate(x=norm_val_X, y=val_Y)\n",
    "\n",
    "print()\n",
    "print(f'loss: {loss}')\n",
    "print(f'wine_quality_loss: {wine_quality_loss}')\n",
    "print(f'wine_type_loss: {wine_type_loss}')\n",
    "print(f'wine_quality_rmse: {wine_quality_rmse}')\n",
    "print(f'wine_type_accuracy: {wine_type_accuracy}')\n",
    "\n",
    "# EXPECTED VALUES\n",
    "# ~ 0.30 - 0.38\n",
    "# ~ 0.30 - 0.38\n",
    "# ~ 0.018 - 0.036\n",
    "# ~ 0.50 - 0.62\n",
    "# ~ 0.97 - 1.0\n",
    "\n",
    "# Example:\n",
    "#0.3657050132751465\n",
    "#0.3463745415210724\n",
    "#0.019330406561493874\n",
    "#0.5885359048843384\n",
    "#0.9974651336669922"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gPtTGAP4usnm"
   },
   "source": [
    "## Analyze the Model Performance\n",
    "\n",
    "Note that the model has two outputs. The output at index 0 is quality and index 1 is wine type\n",
    "\n",
    "So, round the quality predictions to the nearest integer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "tBq9PEeAaW-Y"
   },
   "outputs": [],
   "source": [
    "predictions = model.predict(norm_test_X)\n",
    "quality_pred = predictions[0]\n",
    "type_pred = predictions[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "YLhgTR4xTIxj"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[5.6708837]\n"
     ]
    }
   ],
   "source": [
    "print(quality_pred[0])\n",
    "\n",
    "# EXPECTED OUTPUT\n",
    "# 5.4 - 6.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "MPi-eYfGTUXi"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.00224686]\n",
      "[0.9998627]\n"
     ]
    }
   ],
   "source": [
    "print(type_pred[0])\n",
    "print(type_pred[944])\n",
    "\n",
    "# EXPECTED OUTPUT\n",
    "# A number close to zero\n",
    "# A number close to or equal to 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Kohk-9C6vt_s"
   },
   "source": [
    "### Plot Utilities\n",
    "\n",
    "We define a few utilities to visualize the model performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "62gEOFUhn6aQ"
   },
   "outputs": [],
   "source": [
    "def plot_metrics(metric_name, title, ylim=5):\n",
    "    plt.title(title)\n",
    "    plt.ylim(0,ylim)\n",
    "    plt.plot(history.history[metric_name],color='blue',label=metric_name)\n",
    "    plt.plot(history.history['val_' + metric_name],color='green',label='val_' + metric_name)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6rfgSx7uz5dj"
   },
   "outputs": [],
   "source": [
    "def plot_confusion_matrix(y_true, y_pred, title='', labels=[0,1]):\n",
    "    cm = confusion_matrix(test_Y[1], np.round(type_pred), labels=[0, 1])\n",
    "    disp = ConfusionMatrixDisplay(confusion_matrix=cm,\n",
    "                                  display_labels=[0, 1])\n",
    "    disp.plot(values_format='d');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "dfVLIqi017Vf"
   },
   "outputs": [],
   "source": [
    "def plot_diff(y_true, y_pred, title = '' ):\n",
    "    plt.scatter(y_true, y_pred)\n",
    "    plt.title(title)\n",
    "    plt.xlabel('True Values')\n",
    "    plt.ylabel('Predictions')\n",
    "    plt.axis('equal')\n",
    "    plt.axis('square')\n",
    "    plt.plot([-100, 100], [-100, 100])\n",
    "    return plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "8sd1jdFbwE0I"
   },
   "source": [
    "### Plots for Metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "f3MwZ5J1pOfj"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXhV1b3/8fc3CSEkzElAZtAQBhVRA6JUBEcGkTqL421RtMXptrZqb39W771VO92r1qmoXEodUKsoiKjUAZwQAiLzEAYZJYEwJoGQ5Pv7Yx9CiAk5QsJJTj6v59nPOWetvc9ZZz/6YWWdtdc2d0dERKJXTKQbICIiNUtBLyIS5RT0IiJRTkEvIhLlFPQiIlFOQS8iEuUU9CIiUU5BL/WKma01swIz22Nm35nZeDNrHKobb2ZuZpeUO+axUPm/hV7Hm9lfzGxD6H3WmNn/VvIZB7Ynj+kXFSlDQS/10XB3bwz0Bk4F7i9TtwK46cALM4sDrgRWldnnfiAD6As0AQYBX1f0GWW226v/a4iEJy7SDRCJFHf/zszeJwj8A6YA15tZC3ffDgwGFhAE+gF9gEnuvin0em1oE6mV1KOXesvM2gNDgKwyxXuBycA1odc3AhPKHToL+IWZ/dzMTjYzq/HGihwFBb3UR2+Z2W5gPZAN/K5c/QTgRjNrBpwDvFWu/hHgD8B1QCaw0cxuKrfPW2a2o8x2S7V/C5EwKeilPvqxuzcBBgLdgZSyle7+GZAK/BZ4x90LytUXu/tT7t4faA78HhhnZj3KfUbzMttzNfh9RA5LQS/1lrvPAMYDf66g+kXgl3x/2Kb8exS4+1PAdqBndbdRpDrox1ip7x4D1ppZ73LlTwCfAjPLH2BmdwPzga+A/QRDOE34/swbkVpBQS/1mrvnmNkE4P8Bu8uU5wIfVnJYAfAXIA1wgimZl7v76jL7TDGz4jKvp7v7pdXaeJEwmW48IiIS3TRGLyIS5aoMejPrYGYfm9lSM1tsZndVsI+Z2RNmlmVmC8zstDJ1g81seajuvur+AiIicnjh9OiLgF+6ew+gHzDGzMrPLhgCdA1to4FnAMwsFngqVN8TGFnBsSIiUoOqDHp33+zu80LPdwNLgXbldhsBTPDALKC5mbUhWAsky91Xu3shMDG0r4iIHCM/aNaNmXUmWATqq3JV7QiuMjxgQ6isovIzKnnv0QR/DZCUlHR69+7df0jTjsiCLQto2rApnZt3JjcX1qyBE0+EhIQa/2gRkWo1d+7cre6eWlFd2EEfWsr1DeBud99VvrqCQ/ww5d8vdB8LjAXIyMjwzMzMcJt2xHo904vjWxzPW9e8xRdfQP/+8Mc/wtChNf7RIiLVysy+rawurFk3ZtaAIORfcvc3K9hlA9ChzOv2wKbDlNcKyYnJbCvYBkB6elC2YkUEGyQiUgPCmXVjwAvAUnf/n0p2m0ywCJSZWT9gp7tvBuYAXc2si5nFE6wIOLma2n7Ukhslsy0/CPrkZGjRQkEvItEnnKGb/sANwEIzmx8q+w3QEcDdnwXeBYYSLPeaD/wkVFdkZrcD7wOxwDh3X1yt3+AoJDc62KM3C3r1CnoRiTZVBn1oJb/DrrftweW1Yyqpe5fgH4JaJzkxmdyCXNwdMyM9HT75JNKtEhGpXvX6ytjkRskUlRSxa1/w23J6OqxfD/n5EW6YiEg1qt9Bn5gM8L0fZLOyKjtCRKTuqd9B3ygU9PmaeSMi0at+B325Hn1aWlCuoBeRaFK/g75cj75xY2jXTkEvItGlfgd9uR49aIqliESfeh30LRJaYFhpjx4U9CISfep10MfGxNI8ofn3evTbtgWbiEg0qNdBD4eudwMHZ96sXBmhBomIVDMFfZn1bkBTLEUk+ijoy/Xou3SB2Fj16EUkeijoy/XoGzQIwl49ehGJFgr6Rof26EEzb0QkuijoE5PZU7iHwuLC0rIDQe8V3gtLRKRuUdCXuzoWgqDPz4dNteZeWCIiR05BX8nVsaDhGxGJDgr6Snr0oKAXkeigoK+gR9+uHTRqpKAXkehQ5a0EzWwccDGQ7e4nVVD/K+C6Mu/XA0h191wzWwvsBoqBInfPqK6GV5eKevQxMdC1q4JeRKJDOD368cDgyird/U/u3tvdewP3AzPcPbfMLoNC9bUu5KHiHj1oiqWIRI8qg97dZwK5Ve0XMhJ45ahadIwlNkgkIS7hkB49BEG/ejXs3x+hhomIVJNqG6M3s0SCnv8bZYod+MDM5prZ6Or6rOpW2UVTRUWwdm1k2iQiUl2q88fY4cDn5YZt+rv7acAQYIyZDajsYDMbbWaZZpaZk5NTjc2qWvn1bkAzb0QkelRn0F9DuWEbd98UeswGJgF9KzvY3ce6e4a7Z6SmplZjs6pWfr0bUNCLSPSolqA3s2bAOcDbZcqSzKzJgefAhcCi6vi86lZRjz45GVq2VNCLSN0XzvTKV4CBQIqZbQB+BzQAcPdnQ7tdCnzg7nllDm0NTDKzA5/zsru/V31Nrz4V9ehBM29EJDpUGfTuPjKMfcYTTMMsW7YaOOVIG3YsJTdKJrcgF3cn9A8TEAT9Rx9FsGEiItWg3l8ZC8HQTbEXs3PfzkPK09NhwwbIy6vkQBGROkBBT8VXx8LBH2Szso51i0REqo+CnsNfHQsapxeRuk1BT+U9+rS04FFBLyJ1mYKeynv0SUnQvr2CXkTqNgU9lffoQVMsRaTuU9ADzROaY9j3evSgoBeRuk9BD8TGxNI8oXmlPfrcXNj2/SoRkTpBQR9S0TIIoJk3IlL3KehDKlqqGBT0IlL3KehDkhMrXu+mc2eIi1PQi0jdpaAPqaxH36ABHH+8gl5E6i4FfUhlK1iCZt6ISN2moA9JTkwmb38e+4r2fa8uPT1Y76akJAINExE5Sgr6kNKLpioYvunaFfLzYdOmY90qEZGjp6APSU8Optf8a/W/vl+nmTciUocp6EPO7XIuvVr34tHPHqXEDx2jUdCLSF2moA8xM+7/0f0s3bqUt5e9fUhd27aQmKigF5G6SUFfxpU9ryStZRq///T3uHtpeUxMME6voBeRuqjKoDezcWaWbWaLKqkfaGY7zWx+aHugTN1gM1tuZllmdl91NrwmxMbEcm//e5m7eS7TV08/pC49HZYvj1DDRESOQjg9+vHA4Cr2+dTde4e2/wQws1jgKWAI0BMYaWY9j6axx8INvW6gXZN2PPzpw4eU9+kTTLGcOTNCDRMROUJVBr27zwRyj+C9+wJZ7r7a3QuBicCII3ifY6phXEPuOeseZnw7g8/XfV5aPmYMdOwIP/857N8fwQaKiPxA1TVGf6aZfWNm08zsxFBZO2B9mX02hMoqZGajzSzTzDJzcnKqqVlH5pbTbiElMYVHPnuktCwxER5/HBYvhr/+NYKNExH5gaoj6OcBndz9FOCvwFuhcqtgX6+gLKhwH+vuGe6ekZqaWg3NOnJJ8UncfcbdTF05lfnfzS8tHzEChg6F3/1OF0+JSN1x1EHv7rvcfU/o+btAAzNLIejBdyiza3ugzsTjmL5jaBLfhEc/e7S0zAyeeCIYuvnlLyPYOBGRH+Cog97MjjMzCz3vG3rPbcAcoKuZdTGzeOAaYPLRft6x0jyhOWP6jOG1xa+xYtvBeZUnnAD33QcTJ8JHH0WwgSIiYQpneuUrwJdANzPbYGajzOw2M7sttMsVwCIz+wZ4ArjGA0XA7cD7wFLgNXdfXDNfo2bc3e9uGsY15I+f//GQ8nvvhS5dgh9oCwsj1DgRkTBZ2QuDaouMjAzPzMyMdDMAuOPdO/jb3L+x6s5VdGh2cCRq6lS4+GL4wx/g17+OYANFRAAzm+vuGRXV6crYKtxz1j04zl++/Msh5cOGBT/OPvQQrF9fycEiIrWAgr4KnZp34vpe1zN27lhy8g6d9vnYY+AOv/hFhBonIhIGBX0Y7u1/L3uL9vLYrMcOKe/cGf7jP+Cf/4QPPohM20REqqKgD0P3lO5c3vNyHv/qcRZlH7rkzz33BAue3X477Pv+zalERCJOQR+mxy56jKYNm3LxyxeTnZddWt6wYXCl7MqVwQ+zIiK1jYI+TO2atmPyyMlk52Vz6auXsrdob2ndRRfByJHw4IPw2muRa6OISEUU9D9ARtsMJlw6gS/Wf8EtU245ZM36F16A/v3h+uvhX9+/G6GISMQo6H+gK3pewX8P+m9eXPDiIUsZN2oEkydD9+5w6aVQSy4DEBFR0B+J35z9G67vdT2//fi3/HPJP0vLW7SA996DlJRg8TPdkUpEagMF/REwM54b/hxndTiLGyfdSOamg933tm3h/feD5xdeqFUuRSTyFPRHKCEugUlXT6J149aMmDiCjbs2ltalp8O0abBtGwweDDt2RLChIlLvKeiPQqukVkwZOYXd+3Yz/JXh5BXmldadfjpMmgTLlsHw4VBQEMGGiki9pqA/Sie1OomJV0zkmy3fcM0b17Cv6OBVU+efDy++CJ9/DldfDUVFEWyoiNRbCvpqMLTrUJ4e+jTvrHiHERNHkL8/v7TuqqvgySdhyhS49lrYu/cwbyQiUgPiIt2AaHFrxq3ExcRxy5RbGPbyMKaMnELj+MZAcEPxgoJguYTvvoO33oKWLSPcYBGpN9Sjr0ajThvFi5e9yKfffsqF/7iQnXt3ltb98pfwyivw1VfBhVVr10aunSJSvyjoq9m1J1/Lq1e8SuamTM6bcB7b8reV1l1zDUyfHvTqzzwT5s2LYENFpN5Q0NeAy3tezqSrJ7EoexGD/j6ILXu2lNYNGBD8OBsfHzyfNi2CDRWReiGce8aOM7NsM1tUSf11ZrYgtH1hZqeUqVtrZgvNbL6Z1atFAYalD2PqtVNZtX0V54w/55B59j17wqxZwXz74cPh+ecj2FARiXrh9OjHA4MPU78GOMfdewH/BYwtVz/I3XtXdi/DaHbe8efx3nXvsWn3JgaMH8Ca7WtK69q0gRkz4IIL4JZb4IEHgrtViYhUtyqD3t1nArmHqf/C3beHXs4C2ldT26LC2Z3O5l83/ovtBdvp81wfPlrzUWldkybBQmijRsF//VfQu9+y5TBvJiJyBKp7jH4UUHbU2YEPzGyumY0+3IFmNtrMMs0sMycn53C71jl92/Vl1s2zaN24NRf84wL+8sVfSpc4btAAnnsOnngiWN64Vy94550IN1hEokq1Bb2ZDSII+nvLFPd399OAIcAYMxtQ2fHuPtbdM9w9IzU1tbqaVWukJ6cza9QsLu1+KfdMv4dr37y2dMkEM7jjDpg7NxjSGT48mHufn1/Fm4qIhKFagt7MegHPAyPcvXQ+obtvCj1mA5OAvtXxeXVVk4ZNeP3K13n43Id5ddGrnDXuLFZvX11af+KJwTz7e+6BZ54J1svRFEwROVpHHfRm1hF4E7jB3VeUKU8ysyYHngMXAhXO3KlPzIz7z76fd697l/U715MxNoP3s94vrW/YEP70J/jwQ9i9G844Ax59FIqLI9hoEanTwple+QrwJdDNzDaY2Sgzu83Mbgvt8gCQDDxdbhpla+AzM/sGmA1Mdff3auA71EmD0waTOTqTDs06MOSlITzy6SOH3Jrw3HNhwYLgblX33x+8XrPmMG8oIlIJ81o4py8jI8Mz68m9+PIK87h5ys1MXDSRczqdw98u/hvdUrqV1rvDhAnBGL47/M//wM03B+P6IiIHmNncyqax68rYCEuKT+Lly15m7MVj+WbLN/R6thcPfvJg6XLHZnDTTbBwIfTtC6NHB7cp3LixijcWEQlR0NcCZsYtp9/CsjHLuLzH5Tw04yFOefYUZqydUbpPp07BOjlPPgkzZ8JJJwVr3dfCP8hEpJZR0NcirRu35uXLX+a9696jsLiQgX8fyE/f/mnpwmgxMTBmDMyfHyyjcMMNcPnlkJ0d4YaLSK2moK+FLkq7iEU/X8R9/e/jHwv+QfenujPhmwmUeAkAXbsGvfo//QmmTg2mZb7+unr3IlIxBX0tldggkUfOf4R5o+eR1jKNm966iYyxGbyX9R7uTmxsMN9+3rxgWOeqq4J17mfMqPq9RaR+UdDXcie3PpnPf/o5E348ge17tzPkpSEM/PtAvlj/BRD05mfNCpZRWLcOBg6EIUPg668j224RqT0U9HVAjMVwwyk3sPz25Tw55EmWb11O/3H9Gf7KcBZsWUBcXDDlcuXKYDhn9mw47bTgRicrV0a69SISaQr6OiQ+Np4xfcew6s5VPHzuw3y27jN6P9ub6968jlW5q2jUKBjOWb0afvvbYHG0Hj2CKZmajilSfyno66Ck+CTuP/t+Vt+5mnv738ukpZPo9mQ3bp58M2t3rKVZs2DZ41WrgsXRxo+HtDT4zW9g584q315EooyCvg5r0agFj5z/CKvuXMWYPmN4ccGLdP1rV2575zbW7VxH69bB8sfLlwfTMB95BE44AR5/HPbti3TrReRYUdBHgTZN2vD4kMfJujOL0aeNZtzX4+j6166MmTqGjbs20qVLcHHV3LnQuzfcfXcwpDNxIpSURLr1IlLTFPRRpH3T9jw17Cmy7sziJ71/wth5YznhiRO4c9qdbN69mdNOC66ufe89aNoURo4MllX46KOq31tE6i4FfRTq2Kwjz178LCtuX8H1va7n6TlPc8ITJ/CbD3/Dzn07uOiioHf/978HV9Wedx5ceCF8+WWkWy4iNUFBH8W6tOjC85c8z7Lbl/Hj7j/mkc8e4fjHj+fPX/yZwpICbrwRVqwIpmTOnw9nnQWDBwc3PxGR6KGgrwfSWqbx8uUv8/WtX3NG+zP41fRfkf5kOi/Me4G4+KLSKZl/+ANkZkK/fsEKmbNnR7rlIlIdFPT1SO/jejPtuml8fNPHtG3Slpun3EyvZ3oxaekkkpKcX/8a1q4NZud89VVwd6uLLw7CX0TqLgV9PTSw80BmjZrFG1e9QYmXcNlrl9HvhX5MXzWdpCTnvvuCwP/974Nx+z59giGdjz7SwmkidZGCvp4yMy7rcRmLfr6I54c/z+bdm7nwxQsZ9PdBfL7uc5o0CS6wWrMGHn44GMM/77wg9F97DYqKIv0NRCRc4dwzdpyZZZtZhTf2tsATZpZlZgvM7LQydYPNbHmo7r7qbLhUj7iYOEadNoqVd6zkicFPsGzrMn70fz9i6EtDmbd5Hk2bBvesXbsWxo6FXbvg6quhWzd4+mnIz4/0NxCRqoTTox8PDD5M/RCga2gbDTwDYGaxwFOh+p7ASDPreTSNlZrTMK4hd5xxB6vuXMWj5z3KrA2zOH3s6Vzx2hUsyVlCQgLccgssXQpvvgmpqcFNUDp1gv/8T938RKQ2qzLo3X0mkHuYXUYAEzwwC2huZm2AvkCWu69290JgYmhfqcWS4pO490f3suauNTww4AHeX/U+Jz9zMle+fiVfbfiK2Fi49NJg7H7mzGCGzu9+Bx06BPe2nTs30t9ARMqrjjH6dsD6Mq83hMoqK6+QmY02s0wzy8zJyamGZsnRaJbQjIcGPcSau9bw67N+zfRV0+n3Qj8G/N8AJi+fjFPC2WfDlClBL/+WW4KefkZGMB9/4kTYvz/S30JEoHqC3ioo88OUV8jdx7p7hrtnpKamVkOzpDqkJKbwyPmPsP7f1/PYRY+xbuc6RkwcQY+nejB27lgK9hfQvXtw0/ING+CxxyAnJ1heoVOnYBXNLVsi/S1E6rfqCPoNQIcyr9sDmw5TLnVQk4ZNuKvfXWTdmcXEyyfSJL4Jt75zK50e68RDnzxEbkEuzZrBXXcFq2VOnQqnnAIPPBAM61x3HXzxhaZnikRCdQT9ZODG0OybfsBOd98MzAG6mlkXM4sHrgntK3VYXEwcV590NXNumcMnN33CGe3P4MEZD9LpsU7c/6/7yc7LJiYmuLJ22jRYtgx+9rPgJij9+wd3vnruOcjLi/Q3Eak/zKvoYpnZK8BAIAXYAvwOaADg7s+amQFPEszMyQd+4u6ZoWOHAo8BscA4d/99OI3KyMjwTF2OWWcs3LKQ33/6e15b/BoJcQnclnEbvzrrV7Rp0qZ0nz174KWX4KmnYOFCaNYMfvKT4B+B9PQINl4kSpjZXHfPqLCuqqCPBAV93bRs6zIe/vRhXl74MnExcdx82s3c2/9eOjQ7OILnDp99FszB/+c/gwuvzjsPRo0KZvMkJETwC4jUYQp6OaZW5a7i0c8eZfw34zGMkSePZEjaEAZ1HkTrxq1L9/vuu2AY54UX4NtvoXnzYCz/pz+FU08Fq+jnfBGpkIJeImLdznX84bM/8OLCF9m1bxcAJ6aeyKDOgzi3y7mc0/kcWjZqSUkJfPwxjBsHb7wR3ObwlFOCwL/uOkhOjvAXEakDFPQSUUUlRXy9+Ws+WvMRH6/9mE/XfUr+/nwMo/dxvbmk2yXclnEbxzU+ju3bgzn448YFq2bGxwdDOrfeCgMHqpcvUhkFvdQqhcWFzN44m4/XfMyHaz5k5rczaRDbgJEnjeTufnfT+7jeACxYEAT+hAmwfXvwo+2ttwZX4KqXL3IoBb3Ualm5WTw+63H+b/7/kbc/j4GdB/Lv/f6di9MvJsZiKCgIfrj929/g88+hYUO44oog9H/0I/XyRUBBL3XEjr07eG7uc/x19l9Zv2s9aS3TuOuMu/i33v9G4/jGACxaFAT+hAnBSpo9e8KNNwYranbuHNn2i0SSgl7qlKKSIt5c+ib/O+t/mbVhFk3im3DjKTfys4yfcWKrE4HggqtXXw1m7cyaFRzXr18Q+FdeCe0qXVVJJDop6KXOmrVhFk/PeZrXFr/GvuJ9nN3xbH7e5+dc1uMy4mPjgeDmKK+9FvyIO39+MJQzYEAQ+ldcESypLBLtFPRS523N38r4+eN5JvMZVm9fTaukVow6dRS3nn4rnZp3Kt1v+fKgp//KK8HyC7GxcMEFcO218OMfQ5MmEfwSIjVIQS9Ro8RLmL5qOs9kPsOUFVNwd/q178eQtCEM6TqE09qcRozF4B4stfDKK8H27bfBVbeXXBKE/uDBwY+6ItFCQS9Rad3OdYyfP553VrxD5qZMHKdVUisuOuEihqQN4cITLiQ5MRn34EYpL78c9Pa3bg2uwr3iimA55QEDIC4u0t9G5Ogo6CXq5eTl8P6q95mWNY33s95nW8E2YiyGM9qdwfD04YzoPoIeKT0oKjI+/DAI/UmTgsXWkpODnv5ll8H552u9HambFPRSrxSXFJO5KZNpWdOYunIqmZuC/5ZOaHECI7qN4JJul9C/Y38K98YxbVoQ+O+8Azt3QuPGwRLLl10WPGpMX+oKBb3Uaxt3bWTKiilMXj6ZD9d8SGFxIS0btWRY12GM6DaCIV2HEOeJfPxxcDvEt94KbnbesCGcey4MGRJsaWmR/iYilVPQi4Ts3rebD1Z9wNvL32bqyqnkFuSS2CCRYV2HcWXPKxnadSgJsUl8+WUQ+u+8AytXBsempR0M/YEDoVGjiH4VkUMo6EUqUFRSxMxvZ/L64td5c9mbZOdlk9ggkaFdh3JlzysZ1nUYSfFJrFoV3C1r2jT46CPYuzcYxz/nnGB4Z+hQ9fYl8hT0IlUoLikOQn/J67y59E225G2hUVwjLkq7iKFpQxnSdQjtm7anoABmzID33oN33z3Y209PPxj6AwZo6qYcewp6kR+guKSYz9Z9xutLXmfKiims27kOgJNancSQtCEM7TqU/h360yC2AVlZQU//3XeDNfX37YOkpOCuWcOGBWP8J5yghdek5h110JvZYOBxgnu/Pu/uj5ar/xVwXehlHNADSHX3XDNbC+wGioGiyhpSloJeagt3Z0nOEqZlTWNa1jQ+/fZT9pfsp0l8E84//nyGdR3G8G7DaZXUivz8IOzffRemTg0u0oJg3Z0BA4KhnnPOgW7dFPxS/Y4q6M0sFlgBXABsAOYAI919SSX7Dwf+3d3PDb1eC2S4+9ZwG6ygl9pq977dfLjmQ6atDIJ//a71GMaZHc4snbrZPaU77sESDJ98Egz1zJgR3DoRoHXrg8E/aBD06KHgl6N3tEF/JvCgu18Uen0/gLs/Usn+LwMfu/tzoddrUdBLFHJ3vtnyDW8ve5vJKyYzb/M8ANKT00tD/4x2Z9AgtgHuwXj+gdCfMQM2bAjep3XrIPAPbGlpCn754Y426K8ABrv7zaHXNwBnuPvtFeybSNDrT3P33FDZGmA74MDf3H1sJZ8zGhgN0LFjx9O/PfB3r0gdsX7neiYvn8zby9/mk7WfsL9kPw1jG3Jqm1Pp27Yvfdr1oW+7vqS1TMOIYc2aYKjn44+D2TybNwfv0779wdDv3x+6dlXwS9WONuivBC4qF/R93f2OCva9Grje3YeXKWvr7pvMrBUwHbjD3Wce7jPVo5e6bufenUxfPZ1ZG2Yxe+Ns5m6eS/7+fACaJzQno20Gfdr2oUdKD9KT0+naMp2c9S1KQ/+TTyAnJ3ivlBQ466wg9Pv3h9NP1zIN8n3HbOjGzCYBr7v7y5W814PAHnf/8+E+U0Ev0aaopIilOUuZvXE2czbNYfbG2SzYsoBiLy7dJzUxlfTk9NLgb154Mr7qPOZ8mcDnnx+cyhkfH4T9mWdC377Qpw906aJef313tEEfR/Bj7HnARoIfY69198Xl9msGrAE6uHteqCwJiHH33aHn04H/dPf3DveZCnqpDwqLC1mzfQ3Lty1nxbYVh2yb9wTjOE3im3BJt0u46sSrOLXpRcz9qiGffx7cO3fevGA6JwQLs/XpE2wHwr916wh+OTnmDhf0VS7O6u5FZnY78D7B9Mpx7r7YzG4L1T8b2vVS4IMDIR/SGphkQVcjDni5qpAXqS/iY+PpltKNbindvle3e99uvtzwZelVuy8tfImmDZsyotsIrrrlKv77kQuwkoYsWgSzZ8OcOcH2wQdQUhK8R8eOB0O/b9/grwAt0lY/6YIpkVpuf/F+PlrzEa8tfo1Jyyaxfe92mjVsxrldziWtZRrHtzieE1qcwPEtjic5riMLv2lQGvyzZ8Pq1cH7mAU3U+/bN9hOPRVOOim4wEvqPl0ZKxIlCosL+XD1h7y6+FVmbZjFmh1rKCwuLK2PtVg6NuvI8S2O56RWJ9GnbR/SEvuybUUac+YYs2cH4b81NNnZLJjVc4IWwkAAAAypSURBVMoph27t22vMv65R0ItEqRIvYeOujazevprV21ezavuq0seFWxZSUFQABDN9+rTtE2zt+nJccR82r2jDggXGN98EN1Vfs+bg+7ZoEVzIVX7r1AliYiL0ZeWwFPQi9VBRSRFLcpYEM302zmH2ptks3LKwdKZPcqNkTmx1Ij1TetIztSedknpSsqUnG5Yex8KFxtKlwdW9B6Z5QjCts1u3IPTT04O/Brp2DZ63aBGhLyqAgl5EQvL35zP/u/nM2TiHxTmLWZKzhMU5i9mxd0fpPs0TmtM9pTutk1qT3CiZREumaFcy+duS2bEphey1yaxf2ZJNq1rgBc1hfyPASE4+NPyPP/7glpqqoaCapqAXkUq5O9/t+Y4lOUtKt+XblpOTn8O2/G1sK9h2yO8A5cVZPAk0J7YwCP59O5uzL6c9bMqAjX0g+2QSG8YfEvxdugRb587BY+PGx+77RisFvYgcMXcnb39eaehvy99GbkEuO/buYMfeHWzfu/2Qxx17d7AqdxXbCrYBEEdDUop60yi3D4Vr+7BtQV/2bkwHPzjYn5JyaPCXfezUSXfzCoeCXkSOKXdn7Y61zNk0p/T3gbmb5pK3P7jMplFcIzo06kaK9SApryee04P8b3uQvSyNdWviKSz3B8RxxwWhf2Br2zYoa9Pm4GN9nyaqoBeRiCsuKWbZ1mXM2TSHhVsWsnTrUpZuXcraHWtL94mLiSOtRRqtEjqQUNyKmIJWFO9uxd6trdi1uRVbv23FltWtKNrRGooO7eY3bvz98C+7tW0bPLZsGZ2/FxzVlbEiItUhNiaWE1udyImtTjykPK8wj+XblrMkZwlLc5aybNsyNu3exIZ9q8guyGZPyR5oSbCVOTQxrjHNYluRSCsa7g/+USjZ3ZpNua3Iyj6OHQvbUJB9HOw5DgoPXhIcFxcsGZGSEmxln1f0OiUluKK4Lv/joB69iNRq+fvzyc7LLt227NlCTn5O8DxvyyF1OXk5hywUd0Cj2CSaxhxHYslxxBceR8zeVHxPKkU7W7F3Wyp5Oans2tyK4l2psK8ZFDUEDiZ7gwYH/xFo3hyaNj381qzZ9x8bNarZfyzUoxeROiuxQSKdm3emc/POVe5b4iXkFuTy3Z7vSrfNuzcHz/OC51vylpKTN5OtSVvx1pV3dBtYQ+JIII4ErCSB/KIE8ooasqbYKfb9FLOfYi+ixPfDriLI2w/fFQf/UBS0hPzk4LEgeIzZm0wja05Sw0SaNEqgaaMEmiU1onnjBFo0SaBl0wTapibxi1EdqvHsBRT0IhI1YiyGlMQUUhJTOKnVSYfdt7ikmNyC3OAvgdBfCNl52ewp3MPeor2VbjEWQ1xMHA1iG9AgpgFxMXHE0oCS4jiKCmPYlreTbQW5bC/IZWfhInYX55JXkksJReQBeUB2RQ3aBzErWvELtlT7eVHQi0i9FBsTS2pSKqlJqTX+We7O7sLd7Ni7g71FeynYX1D6D0dBUQEF+/eyY89eCvfVzPoSCnoRkRpmZjRt2JSmDZtG5PO1PJGISJRT0IuIRDkFvYhIlFPQi4hEubCC3swGm9lyM8sys/sqqB9oZjvNbH5oeyDcY0VEpGZVOevGzGKBp4ALgA3AHDOb7O5Lyu36qbtffITHiohIDQmnR98XyHL31e5eCEwERoT5/kdzrIiIVINwgr4dsL7M6w2hsvLONLNvzGyamR1YeijcYzGz0WaWaWaZOWXvXSYiIkclnKCvaBme8gtEzAM6ufspwF+Bt37AsUGh+1h3z3D3jNTUmr9STUSkvggn6DcAZVfZaQ9sKruDu+9y9z2h5+8CDcwsJZxjRUSkZoUT9HOArmbWxczigWuAyWV3MLPjzIIFOM2sb+h9t4VzrIiI1KwqZ924e5GZ3Q68D8QC49x9sZndFqp/FrgC+JmZFQEFwDUeLHRf4bE19F1ERKQCuvGIiEgUONyNR3RlrIhIlFPQi4hEOQW9iEiUU9CLiEQ5Bb2ISJRT0IuIRDkFvYhIlFPQi4hEOQW9iEiUU9CLiEQ5Bb2ISJRT0IuIRDkFvYhIlFPQi4hEOQW9iEiUU9CLiEQ5Bb2ISJRT0IuIRLmwgt7MBpvZcjPLMrP7Kqi/zswWhLYvzOyUMnVrzWyhmc03M90fUETkGKvy5uBmFgs8BVwAbADmmNlkd19SZrc1wDnuvt3MhgBjgTPK1A9y963V2G4REQlTOD36vkCWu69290JgIjCi7A7u/oW7bw+9nAW0r95miojIkQon6NsB68u83hAqq8woYFqZ1w58YGZzzWz0D2+iiIgcjSqHbgCroMwr3NFsEEHQ/6hMcX9332RmrYDpZrbM3WdWcOxoYDRAx44dw2iWiIiEI5we/QagQ5nX7YFN5Xcys17A88AId992oNzdN4Ues4FJBENB3+PuY909w90zUlNTw/8GIiJyWOEE/Rygq5l1MbN44BpgctkdzKwj8CZwg7uvKFOeZGZNDjwHLgQWVVfjRUSkalUO3bh7kZndDrwPxALj3H2xmd0Wqn8WeABIBp42M4Aid88AWgOTQmVxwMvu/l6NfBMREamQuVc43B5RGRkZnpmpKfciIuEys7mhDvb36MpYEZEop6AXEYlyCnoRkSinoBcRiXIKehGRKKegFxGJcgp6EZEop6AXEYlyCnoRkSinoBcRiXIKehGRKKegFxGJcgp6EZEop6AXEYlyCnoRkSinoBcRiXIKehGRKKegFxGJcgp6EZEoF1bQm9lgM1tuZllmdl8F9WZmT4TqF5jZaeEeKyIiNavKoDezWOApYAjQExhpZj3L7TYE6BraRgPP/IBjRUSkBoXTo+8LZLn7ancvBCYCI8rtMwKY4IFZQHMzaxPmsSIiUoPiwtinHbC+zOsNwBlh7NMuzGMBMLPRBH8NAOwxs+VhtK0iKcDWIzy2pqltR0ZtOzJq25Gpq23rVNlB4QS9VVDmYe4TzrFBoftYYGwY7TksM8t094yjfZ+aoLYdGbXtyKhtRyYa2xZO0G8AOpR53R7YFOY+8WEcKyIiNSicMfo5QFcz62Jm8cA1wORy+0wGbgzNvukH7HT3zWEeKyIiNajKHr27F5nZ7cD7QCwwzt0Xm9ltofpngXeBoUAWkA/85HDH1sg3Oeioh39qkNp2ZNS2I6O2HZmoa5u5VzhkLiIiUUJXxoqIRDkFvYhIlIuaoK/NSy2Y2VozW2hm880ssxa0Z5yZZZvZojJlLc1supmtDD22qEVte9DMNobO33wzGxqBdnUws4/NbKmZLTazu0LlET9vh2lbbThvCWY228y+CbXtoVB5bThvlbUt4uetTBtjzexrM3sn9PqIzltUjNGHllpYAVxAMNVzDjDS3ZdEtGEhZrYWyHD3WnERhpkNAPYQXM18Uqjsj0Cuuz8a+oeyhbvfW0va9iCwx93/fKzbU6ZdbYA27j7PzJoAc4EfA/9GhM/bYdp2FZE/bwYkufseM2sAfAbcBVxG5M9bZW0bTITP2wFm9gsgA2jq7hcf6f+n0dKj11ILP4C7zwRyyxWPAP4eev53gqA45ippW8S5+2Z3nxd6vhtYSnDld8TP22HaFnGhZVH2hF42CG1O7ThvlbWtVjCz9sAw4PkyxUd03qIl6CtbgqG2cOADM5sbWuqhNmoduvaB0GOrCLenvNstWBl1XKSGlQ4ws87AqcBX1LLzVq5tUAvOW2j4YT6QDUx391pz3ippG9SC8wY8BvwaKClTdkTnLVqCPuylFiKkv7ufRrCK55jQ8ISE7xngBKA3sBn4S6QaYmaNgTeAu919V6TaUZEK2lYrzpu7F7t7b4Ir4/ua2UmRaEdFKmlbxM+bmV0MZLv73Op4v2gJ+nCWaYgYd98UeswGJhEMNdU2W0JjvQfGfLMj3J5S7r4l9D9kCfAcETp/oXHcN4CX3P3NUHGtOG8Vta22nLcD3H0H8AnBGHitOG8HlG1bLTlv/YFLQr/vTQTONbMXOcLzFi1BX2uXWjCzpNAPZJhZEnAhsOjwR0XEZOCm0PObgLcj2JZDHPgPO+RSInD+Qj/cvQAsdff/KVMV8fNWWdtqyXlLNbPmoeeNgPOBZdSO81Zh22rDeXP3+929vbt3Jsizj9z9eo70vLl7VGwESzCsAFYB/xHp9pRp1/HAN6FtcW1oG/AKwZ+k+wn+GhoFJAMfAitDjy1rUdv+ASwEFoT+Q28TgXb9iGA4cAEwP7QNrQ3n7TBtqw3nrRfwdagNi4AHQuW14bxV1raIn7dy7RwIvHM05y0qpleKiEjlomXoRkREKqGgFxGJcgp6EZEop6AXEYlyCnoRkSinoBcRiXIKehGRKPf/AX3pzxsZT99EAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_metrics('wine_quality_root_mean_squared_error', 'RMSE', ylim=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "QIAxEezCppnd"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAEICAYAAABWJCMKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXRV9bn/8feTOYEACQYSBpkVEJVgAMUplFoBrWDFqtcBrZZSpa1VV2t/7VXs8nZZW7SDXoeqLa0DpSKVVhy4SqytAgkQgRCGgAyBhEQgISEJIcnz++O7Ew4xJCcB3EnO81prr3POd0/P3q35sKfvFlXFGGNM6AnzuwBjjDH+sAAwxpgQZQFgjDEhygLAGGNClAWAMcaEKAsAY4wJURYApkMTkUtFZLPfdRjTEVkAmHZFRH4iIksbtW09QduNqvqRqp59ims4U0TKAwYVkcMBvy89les7QQ1zReTl070eE9oi/C7AmEb+BTwoIuGqWisiyUAkMKZR21Bv2lNOVXcBXet/i4gC56tq3ulYnzF+sSMA095k4v7gj/Z+XwYsBzY3atumqntFJF1E8utnFpEdIvKAiKwTkVIR+auIxASMv1pEskWkREQ+FpHzgi1MRJJFpEJEega0XSAixSISKSK3i8h/ROT33ro3icikgGm7i8iLIlIgIntE5FERCW/tDhKRa0Qkx9uGDBEZETDux96yy0Rkc/36RWSciGSJyCER2SciT7R2vabzsQAw7YqqVgMrcX/k8T4/Av7dqK25f/1/E5gMDALOA24HEJExwEvAd4CewHPAEhGJDrK2QiDDW369W4AFqnrU+z0e2A6cATwMvCEiid64+UAN7uglFfgacFcw664nImcBrwH3AknAUuAfIhIlImcDc4CxqhoPXAns8Gb9LfBbVe0GDAEWtma9pnOyADDt0Ycc+2N/KS4APmrU9mEz8/9OVfeq6gHgHxw7cvg28JyqrlTVWlWdDxwBLmxFbfNxf/Tx/vV+E/CXgPFFwG9U9aiq/hV35HKViPQGpgD3quphVS0CngRubMW6AW4A3lLVZV7o/BqIBSYAtUA0MFJEIlV1h6pu8+Y7CgwVkTNUtVxVV7RyvaYTsgAw7dG/gEtEJAFIUtWtwMfABK9tFM0fARQGfK/g2Pn8AcD93qmTEhEpAfoDfVpR25u4P7CDgSuAUlVdFTB+jx7fw+JOb/kDcKe2CgLW/RzQqxXrxlvWzvofqloH7Ab6etco7gXmAkUiskBE6rftTuAsYJOIZIrI1a1cr+mELABMe/QJ0B2YBfwHQFUPAXu9tr2q+lkblrsb+B9V7REwxKnqa8EuQFWrcKdPbgZu5fh//QP0FREJ+H2mV/du3NHGGQHr7qaq57RyG/biwgQAb139gT1efa+q6iXeNAr80mvfqqo34QLnl8DrItKlles2nYwFgGl3VLUSyALuw536qfdvr62td//8AZgtIuPF6SIiV4lIfCuX82fcdYVrgMa3avYCvu9dFL4eGAEsVdUC4D1gnoh0E5EwERkiIpc3s54wEYkJGKJx4XOViEwSkUjgflywfCwiZ4vIV7zpqoBK3GkhROQWEUnyjhhKvOXXtnK7TSdjAWDaqw9xf0z/HdD2kdfWpgBQ1SzcdYCngINAHt4F4lYu5z9AHbBGVXc0Gr0SGAZ8DvwPMENV93vjbgOigI3e+l8HUppZ1U24P+L1wzZV3Yy7BvF7bx1fB77uXTyPBh7z2gtx++r/ecuaDOSISDnugvCN3tGMCWFiL4QxpvVE5APgVVV9IaDtduAu7xSMMe2ePQhmTCuJyFhgDDDN71qMORlBnQISkcneQyV5IvJgE+Nv9h68Wec9XHN+S/OKSKKILBP3SP8y7+4OY9o1EZkP/B/uds4yv+sx5mS0eArIu9d5C+6Wt3zck5o3qerGgGkmALmqelBEpgBzVXV8c/OKyOPAAVV9zAuGBFX98WnYRmOMMU0I5ghgHJCnqtu9C00LaHToq6ofq+pB7+cKoF8Q807DPVSD9zm97ZthjDGmtYK5BtAXdw9zvXzc4+4ncifwdhDz9vZujUNVC0SkyQdiRGQW7t5vunTpcsHw4cODKLnzWFu4ljNiz6B/9/7s2AGlpXD++S3OZowxDVavXv25qiY1bg8mAKSJtibPG4nIRFwA1N8FEfS8J6KqzwPPA6SlpWlWVlZrZu/wxjw3hl5devHOLe/wxBNw//3w9tuQ9IX/KY0xpmkisrOp9mBOAeXjnjSs1w/3NGLjFZwHvABMC7jvubl594lIijdvCq4PFdPIqF6jWLdvnfs+yrVt2OBjQcaYTiOYAMgEhonIIBGJwnVetSRwAhE5E3gDuFVVtwQ57xJgpvd9Jq6PFdNIanIqBeUF7Cvfx7nnujYLAGPMqdBiAKhqDa6L2XeBXGChquaIyGwRme1N9hCue93/FdfXelZz83rzPAZcISJbcXcJPXYKt6vTSE1JBdy1gORkSEyE9et9LsoY0ykE9SCYqi7F9Tse2PZswPe7OEG/5k3N67XvByZ9cQ4TaHSy68k4uzCbyUMnc+65dgRgjDk1rC+gdq5HTA8G9hjI2sK1gLsOsGEDWA8expiTZQHQAaQmp7K2wAXAuedCWRns2uVzUcaYDs8CoANITU5l64GtlB0pa7gTyK4DGGNOlgVAB1B/HWDdvnV2K6gx5pSxAOgAAu8E6t4d+ve3IwBjzMmzAOgA+sb35Yy4M467DmBHAMaYk2UB0AGICKnJqWTvywbcnUC5uXD0qM+FGWM6NAuADmJ08mg2FG3gaO1Rzj3X/fHfutXvqowxHZkFQAeRmpxKdW01G4s32p1AxphTwgKgg6i/EJxdmM3w4RAebtcBjDEnxwKggxiWOIy4yDjWFq4lJgaGDbMjAGPMybEA6CDCw8I5r/d5X+gSwhhj2soCoANJTU4luzCbOq3j3HNh+3Y4fNjvqowxHZUFQAeSmpzKoSOH2FGyg1GjXIdwGzf6XZUxpqOyAOhA6ruEWFuw1l4OY4w5aRYAHci5vc8lXMJZW7iWwYMhNtYuBBtj2s4CoAOJiYhhRNIIsguzCQ+HkSPtCMAY03YWAB1ManLqcXcC2RGAMaatggoAEZksIptFJE9EHmxi/HAR+UREjojIAwHtZ3vvCK4fDonIvd64uSKyJ2Dc1FO3WZ3X6OTR7C3bS9HhIs49FwoL4fPP/a7KGNMRtRgAIhIOPA1MAUYCN4nIyEaTHQC+D/w6sFFVN6vqaFUdDVwAVACLAyZ5sn689+5g04LUZK9r6IK19m4AY8xJCeYIYByQp6rbVbUaWABMC5xAVYtUNRNorn/KScA2Vd3Z5mrNcS+JtzuBjDEnI5gA6AvsDvid77W11o3Aa43a5ojIOhF5SUQS2rDMkJMQm8CA7gNYW7iWlBTo2RPWrPG7KmNMRxRMAEgTbdqalYhIFHAN8LeA5meAIcBooACYd4J5Z4lIlohkFRcXt2a1nVZqirsQLAIXXQSffOJ3RcaYjiiYAMgH+gf87gfsbeV6pgBrVHVffYOq7lPVWlWtA/6AO9X0Bar6vKqmqWpaUlJSK1fbOaUmp7J1/1bKq8uZMAE2bYL9+/2uyhjT0QQTAJnAMBEZ5P1L/kZgSSvXcxONTv+ISErAz2sBO5MdpNTkVBRl3b51XHSRa1uxwt+ajDEdT4sBoKo1wBzgXSAXWKiqOSIyW0RmA4hIsojkA/cBPxORfBHp5o2LA64A3mi06MdFZL2IrAMmAj88ZVvVyQV2CTF2rHs3wMcf+1yUMabDiQhmIu8WzaWN2p4N+F6IOzXU1LwVQM8m2m9tVaWmQb9u/egZ25Pswmy6jIPRo+06gDGm9exJ4A5IRBouBANMmAArV0JNjc+FGWM6FAuADmp079GsL1rP0dqjXHQRVFTAunV+V2WM6UgsADqo1BT3kvjcz3OZMMG12XUAY0xrWAB0UPVdQmQXZnPmmdCnjwWAMaZ1LAA6qLN6nkVsRCxrC9wDYRMm2IVgY0zrWAB0UI1fEj9hAuzYAXtb+4ieMSZkWQB0YPUviVfVhgfC7CjAGBMsC4AOLDUlldIjpewo2UFqKkRH23UAY0zwLAA6sIYnggvXEh0NaWl2BGCMCZ4FQAd2bi/3kvg1Ba4/6AkTYPVqqKryuTBjTIdgAdCBxUbGkpqSyke7PgJcAFRX2/sBjDHBsQDo4NIHpLMifwWVRysbLgTbdQBjTDAsADq49IHpVNdWsyJ/Bb17w+DBFgDGmOBYAHRwlw64lDAJY/mO5cCxB8K0Ve9sM8aEIguADq5bdDcuSLmAjB0ZgAuAwkL3UJgxxjTHAqATSB+Yzso9K6k4WmHXAYwxQbMA6AQCrwOMGgVdu1oAGGNaZgHQCVxy5iWESzgZOzKIiIDx4y0AjDEtswDoBLpFd+OCPhccdyF43TooL/e5MGNMuxZUAIjIZBHZLCJ5IvJgE+OHi8gnInJERB5oNG6H9/L3bBHJCmhPFJFlIrLV+0w4+c0JXekD0lmZ764DTJgAdXWwapXfVRlj2rMWA0BEwoGngSnASOAmERnZaLIDwPeBX59gMRNVdbSqpgW0PQi8r6rDgPe936aN0gemc7TuKJ/s/oTx412bnQYyxjQnmCOAcUCeqm5X1WpgATAtcAJVLVLVTOBoK9Y9DZjvfZ8PTG/FvKaRi8+8uOE6QEICjBxpAWCMaV4wAdAX2B3wO99rC5YC74nIahGZFdDeW1ULALzPXk3NLCKzRCRLRLKKi4tbsdrQ0tR1gE8+caeCjDGmKcEEgDTR1prnTC9W1TG4U0j3iMhlrZgXVX1eVdNUNS0pKak1s4aciQMnsmrPKg5XH2bCBCgpgc2b/a7KGNNeBRMA+UD/gN/9gKBfPKiqe73PImAx7pQSwD4RSQHwPouCXaZpWsN1gPxP7IEwY0yLggmATGCYiAwSkSjgRmBJMAsXkS4iEl//HfgasMEbvQSY6X2fCbzZmsLNF13c/9h1gLPOgsRECwBjzIlFtDSBqtaIyBzgXSAceElVc0Rktjf+WRFJBrKAbkCdiNyLu2PoDGCxiNSv61VVfcdb9GPAQhG5E9gFXH9qNy30xEfHk9YnjeU7lhMWBhddZAFgjDmxFgMAQFWXAksbtT0b8L0Qd2qosUPA+SdY5n5gUtCVmqCkD0xn3ifzvOsAXXjrLThwwB0NGGNMIHsSuJOZOHAiNXU1fLz7YyZMcG12FGCMaYoFQCcT+DzA+PEQFwdLl7Y8nzEm9FgAdDJdo7oytu9YMnZmEBsLV14JS5bYC2KMMV9kAdAJpQ9IZ9WeVZRXlzNtGuzZA6tX+12VMaa9sQDohNIHpjdcB7jqKggLgzftJltjTCMWAJ3QxWdeTERYBBk7MjjjDLjkEvj73/2uyhjT3lgAdEJdo7oyts/YhvcET5sGGzbA9u3+1mWMaV8sADqp9IHpZO7NbLgOAHYayBhzPAuATqr+OsB/dv2HIUPgnHMsAIwxx7MA6KQm9J/QcB0A3Gmgjz6C/fv9rcsY035YAHRSXaO6Mq7vODJ2ZgAwfbp7N8Bbb/lblzGm/bAA6MTSB6STuSeTsiNlXHAB9Oljp4GMMcdYAHRi6QPTqdVaPtr1EWFhcM018O67UFXld2XGmPbAAqATu+TMS+gS2YUlm93rG6ZNg8OH4f33fS7MGNMuWAB0YrGRsVx11lW8kfsGtXW1TJwI8fF2GsgY41gAdHIzRsyguKKYj3Z9RHQ0TJ4M//iHvSzeGGMB0OlNGTaF2IhYFm1cBLjTQIWFsGqVz4UZY3xnAdDJdY3qyuShk1mUu4g6rWPqVAgPt9NAxpggA0BEJovIZhHJE5EHmxg/XEQ+EZEjIvJAQHt/EVkuIrkikiMiPwgYN1dE9ohItjdMPTWbZBqbMXIGBeUFfLL7ExIS4PLLLQCMMUEEgIiEA08DU3Aver9JREY2muwA8H3g143aa4D7VXUEcCFwT6N5n1TV0d5g7606Ta4+62qiwqNYlOtOA02fDrm5sHWrz4UZY3wVzBHAOCBPVberajWwAJgWOIGqFqlqJnC0UXuBqq7xvpcBuUDfU1K5CVq36G58bcjXeH3j66gq11zj2u0owJjQFkwA9AV2B/zOpw1/xEVkIJAKrAxoniMi60TkJRFJOMF8s0QkS0SyiouLW7ta45kxYga7D+0mc28mAwbA6NEWAMaEumACQJpoa9UbZkWkK7AIuFdVD3nNzwBDgNFAATCvqXlV9XlVTVPVtKSkpNas1gS45uxriAiL4PWNrwPubqCPPwbLVGNCVzABkA/0D/jdD9gb7ApEJBL3x/8VVX2jvl1V96lqrarWAX/AnWoyp0lCbAJfHfxVFuUuQlWZNs09C/DPf/pdmTHGL8EEQCYwTEQGiUgUcCOwJJiFi4gALwK5qvpEo3EpAT+vBTYEV7Jpq+tGXMf2g9vJLsxm9Gg480w7DWRMKGsxAFS1BpgDvIu7iLtQVXNEZLaIzAYQkWQRyQfuA34mIvki0g24GLgV+EoTt3s+LiLrRWQdMBH44anfPBNo+vDphEs4r298HRHXOdx770FFhd+VGWP8IKqtOp3vq7S0NM3KyvK7jA7tq3/+KrsP7WbTPZv48ENh4kR4+WW4+Wa/KzPGnC4islpV0xq325PAIWbGyBls2b+FnOIcLrsMhgyBF17wuypjjB8sAELM9OHTEYTXN75OWBjceSdkZNhDYcaEIguAEJPcNZlLB1za8FTwzJmub6CXXvK5MGPMl84CIATNGDGDDUUb2PT5Jvr0gauugj/9CY4ebXFWY0wnYgEQgr4x4hsADV1E33WX6yJ6qfXGZExIsQAIQX279eWifhfxeq57KnjKFEhJsYvBxoQaC4AQNWPkDLILs9l2YBsREXDHHe4IYM8evyszxnxZLABC1HUjrgNouBj8rW+5riHmz/ezKmPMl8kCIEQN6DGAtD5pDZ3DDRkCEyfCiy/a+4KNCRUWACFsxogZZO7NZPvB7YC7GLx9u3suwBjT+VkAhLCbz7uZcAnnuaznAPjGNyAhwS4GGxMqLABCWL9u/Zg+fDovrH2ByqOVxMTALbfAokWwf7/f1RljTjcLgBA3Z9wcDlQe4K85fwXcaaDqanjlFZ8LM8acdhYAIe7yAZdzTtI5PLXqKVSV886DsWPhD3+ADtRRrDGmDSwAQpyIcM/Ye1hdsJpVe1YB7ihgwwbIzPS5OGPMaWUBYLjlvFuIj4rnqcynALjxRoiLs4vBxnR2FgCG+Oh4bh99OwtzFlJ0uIhu3eCGG+C116C83O/qjDGniwWAAeDusXdTXVvNC2vcP/vvusv98V+40OfCjDGnTVABICKTRWSziOSJyINNjB8uIp+IyBEReSCYeUUkUUSWichW7zPh5DfHtNXwM4bz1cFf5ZmsZ6ipq+Gii2D4cHcx2BjTObUYACISDjwNTAFGAjeJyMhGkx0Avg/8uhXzPgi8r6rDgPe938ZH94y9h/xD+fxj8z8Qge9+F1asgOXL/a7MGHM6BHMEMA7IU9XtqloNLACmBU6gqkWqmgk0fqVIc/NOA+q7HpsPTG/jNphT5OqzrubM7mc2XAz+9rehTx/42c/sllBjOqNgAqAvsDvgd77XFozm5u2tqgUA3mevphYgIrNEJEtEsoqLi4NcrWmLiLAIZl8wmw8++4Dc4lxiY+G//xs+/hjeftvv6owxp1owASBNtAX778GTmddNrPq8qqapalpSUlJrZjVtcNeYu4gKj+LpzKcB1030oEF2FGBMZxRMAOQD/QN+9wP2Brn85ubdJyIpAN5nUZDLNKdRUpckbjjnBuZ/Op9DRw4RFQUPPwxr18Ibb/hdnTHmVAomADKBYSIySESigBuBJUEuv7l5lwAzve8zgTeDL9ucTnPGzaG8upy/fPoXwHUQN3w4PPQQ1Nb6XJwx5pRpMQBUtQaYA7wL5AILVTVHRGaLyGwAEUkWkXzgPuBnIpIvIt1ONK+36MeAK0RkK3CF99u0A+P6jmNsn7E8nfk0qkp4OPz857Bxo3s4zBjTOYh2oBO7aWlpmpWV5XcZIWF+9nxuf/N23r/tfb4y6CvU1cGYMVBWBps2QWSk3xUaY4IlIqtVNa1xuz0JbJp0w6gb6Bnbk6dWuVtCw8Lg0UfdG8P+9Cd/azPGnBoWAKZJMRExfOeC7/D3TX8nuzAbgKuuggsvdKeDqqp8LtAYc9IsAMwJPTDhARJjE7nv3ftQVUTcUUB+Pjz3nN/VGWNOlgWAOaGE2AQeSX+E5TuWs2Szu3lr0iSYOBF+8Qs4fNjnAo0xJ8UCwDTrO2nfYWTSSB5Y9gDVtdWAOwooKoLf/97n4owxJ8UCwDQrIiyCeV+bR96BvIYLwhMmwNSp8PjjUFrqc4HGmDazADAtmjx0MlOGTuHnH/6c4sOuP6ZHH4WDB2HePJ+LM8a0mQWACcq8r82jvLqcuRlzAUhNda+O/OUvISen+XmNMe2TBYAJyoikEXw37bs8t/o5corcX/zf/ha6dYOZM+Fo447AjTHtngWACdrc9LnER8dz/3v3A9CrFzz7LKxeDY9ZRx7GdDgWACZoPeN68vDlD/Putnd5e6t7QcB118F//Zd7OGztWp8LNMa0igWAaZW7x97NsMRh3PfefRytded9fv97SEqC226DI0d8LtAYEzQLANMqUeFRzPvaPDZ9vonnVrvHgRMT4YUXYMMGeOQRnws0xgTNAsC02tVnXc2kQZN4OONhDlYeBNxzAXfe6e4KWrHC5wKNMUGxADCtJiI8ceUTlFSV8NDyhxran3gC+vVzdwVVVPhYoDEmKBYApk3O630e94y9h6cyn+KfW/4JuFtC//hH2LIFfvpTnws0xrTIAsC02eNXPM6YlDHctvg2dpTsAOArX4E5c+A3v4GMDF/LM8a0wALAtFlMRAx/u/5v1Gkd3/zbNzlS424BeuwxGDoU7rjDvUHMGNM+BRUAIjJZRDaLSJ6IPNjEeBGR33nj14nIGK/9bBHJDhgOici93ri5IrInYNzUU7tp5sswOGEwf5z2RzL3ZvLAew8A0KULzJ8Pu3bBt78NdXU+F2mMaVKLASAi4cDTwBRgJHCTiIxsNNkUYJg3zAKeAVDVzao6WlVHAxcAFcDigPmerB+vqktPemuML64dcS33XXgfT2U+xcKchYDrMfQXv4C//hV+9COfCzTGNCmYI4BxQJ6qblfVamABMK3RNNOAP6uzAughIimNppkEbFPVnSddtWl3HvvqY1zU7yLuWnIXW/ZvAdwf/jlzXI+hTz7pc4HGmC8IJgD6ArsDfud7ba2d5kbgtUZtc7xTRi+JSEJTKxeRWSKSJSJZxcXFQZRr/BAZHslfZ/yVqPAoZiycQeXRSkTcxeDrroP77oMFC/yu0hgTKJgAkCbatDXTiEgUcA3wt4DxzwBDgNFAAdBkz/Kq+ryqpqlqWlJSUhDlGr/0796fl7/xMhuKNjBn6RwAwsPh5Zfh0kvd8wHLl/tcpDGmQTABkA/0D/jdD9jbymmmAGtUdV99g6ruU9VaVa0D/oA71WQ6uMlDJ/PTS3/KS9kv8afsPwEQEwNvvunuDJo+HT791N8ajTFOMAGQCQwTkUHev+RvBJY0mmYJcJt3N9CFQKmqFgSMv4lGp38aXSO4FtjQ6upNuzQ3fS4TB07k7rfuZt2+dQAkJMA770B8PEyZAjvtSpAxvmsxAFS1BpgDvAvkAgtVNUdEZovIbG+ypcB2IA/3r/m76+cXkTjgCuCNRot+XETWi8g6YCLww5PdGNM+hIeF8+p1r9IjpgdXvnwlucW5APTvD2+/7bqJmDIFDhzwuVBjQpyoNj6d336lpaVpVlaW32WYIOUW5zJx/kQAPpj5ASOT3N3DGRlw5ZWQlgbLlkFcnI9FGhMCRGS1qqY1brcngc1pMyJpBBm3ZyAiTJw/seFVkunp7sLwJ5+4riP27Wt+OcaY08MCwJxWw88YTsbMDMIlnInzJ7KhyF3quf56WLQI1q2D8ePduwSMMV8uCwBz2p19xtlk3J5BZHgkE+dPZP2+9QBcey38619QXe2eHH7nHZ8LNSbEWACYL8VZPc8iY2YG0eHRTJw/kU8L3b2gaWmwahUMGQJXXQVPPeVzocaEEAsA86UZ1nMYGbdnEBsZy6Q/TyK7MBtwL5H56CMXAN/7nhtqanwu1pgQYAFgvlRDE4eSMTODuMg4Jv15EivzVwLQtSssXuy6jHjqKbjmGjh0yOdijenkLADMl25I4hAybs+gR0wPLv/T5by87mXAdRsxbx48+yy89x5cfDHk5vpcrDGdmAWA8cXghMGsumsVE/pP4NbFt/KjZT+itq4WgO98xz0wVlAAY8a4I4IO9LiKMR2GBYDxTc+4nrx7y7vcnXY3v/r4V1yz4BpKq0oBuOIKWL8eJk501wQmT4a9jXugMsacFAsA46vI8EievuppnrnqGd7b9h4XvnghW/dvBSAlBd56C/73f91F4lGj4G9/a2GBxpigWQCYdmF22mz+79b/o/hwMeNeGMeybcsAEIHvfhfWrnW9iX7zm3DrrVBa6nPBxnQCFgCm3bh84OVkfjuTft36MfmVyfxmxW+o76vq7LPhP/+Bhx+G116D886zdwsYc7IsAEy7MihhEB9/62O+ftbX+eG7PyR9fnpD9xGRkTB3rguCqCjXj9CMGbB1q781G9NRWQCYdic+Op43bniD565+jg1FGxj97Gjuf/d+Dh1xDwaMHw/Z2fDII677iJEj3YVie2OoMa1jAWDapTAJY9YFs9gyZwt3pt7JkyueZPhTw3l1/auoKl26wEMPwbZt8O1vwzPPuO4kfvEL974BY0zLLABMu9YzrifPff05Vty1gr7d+nLzGzcf17V0797uLqENG9wpoZ/+FM46C/74R6it9bl4Y9o5CwDTIYzrO44Vd67guaufY33Res5/9nx++M4PKT7szvsMHw5//7vrXbRfP/jWt2DYMPjxj2H1anuQzJim2BvBTIezv2I/P3n/J7y49kViI2L53rjv8cCEB+gZ1xNwf+wXLYIXXoD333cdyw0a5N5BcP31cMEF7vZSY0LFSb0RTEQmi8hmEckTkQebGC8i8jtv/DoRGRMwbof37t9sEY5OqREAABFQSURBVMkKaE8UkWUistX7TGjrxpnQ0jOuJ89//Xly7s7hmrOv4Zf/+SUDfzuQ//7gvzlYeRARd3fQO++4t429+KK7jfSJJ2DsWHet4Ec/gs2b/d4SY/zVYgCISDjwNDAFGAncJCIjG002BRjmDbOAZxqNn6iqoxsl0IPA+6o6DHjf+21M0IafMZxXr3uV9d9dz5ShU3j0o0cZ+NuBzM2YS0lVCQCJie500NtvHx8GTz7p7h669Va7jdSErmCOAMYBeaq6XVWrgQXAtEbTTAP+rM4KoIeIpLSw3GnAfO/7fGB6K+o2psE5vc5h4fUL+XT2p0waNIlHPnyEQb8dxEPLH2JX6a6G6QLDYM8e1/X0okXu+sHMmZCX5+NGGOODYAKgL7A74He+1xbsNAq8JyKrRWRWwDS9VbUAwPvs1dTKRWSWiGSJSFax3ehtmnFe7/N444Y3WDNrDZcNuIxH//Uog347iKtfvZolm5dQU3fsLTO9esGvfgWffQb33gsLF7oguOMO2L7dx40w5ksUTAA0dbms8ZXj5qa5WFXH4E4T3SMil7WiPlT1eVVNU9W0pKSk1sxqQlRqSipv3vgm23+wnZ9c8hPWFKxh2oJpDPjNAB5a/hA7S3Y2TNu7t3sHwWefuYfJFixwt5Heead74tjeTGY6s2ACIB/oH/C7H9C4Y94TTqOq9Z9FwGLcKSWAffWnibzPotYWb0xzBvYYyKNfeZSd9+5k8Q2LOb/3+Q1HBVNfmcpr619r6H46OdldF9i+He65B155BS65xB0p3HQT/OUv9qSx6XxavA1URCKALcAkYA+QCfyXquYETHMVMAeYCowHfqeq40SkCxCmqmXe92XAz1X1HRH5FbBfVR/z7ixKVNUfNVeL3QZqTtbOkp28uPZFXlz7InvL9hIZFsmkwZOYfvZ0pg2fRnLXZABKSmDZMli69NgFZBEYNw6mToUpU9zLasLDfd4gY4JwottAg3oOQESmAr8BwoGXVPV/RGQ2gKo+KyICPAVMBiqAO1Q1S0QG4/7VDxABvKqq/+MtsyewEDgT2AVcr6oHmqvDAsCcKrV1tazcs5LFuYtZvGkx2w5uQxAu7Hch1w6/lunDpzOs5zAA6upcd9RvveUCYdUq96xB9+5w6aWQnu6G0aMtEEz7dFIB0F5YAJjTQVXJKc7h75v+zuJNi1lTsAaAQT0GkT4wvWE4s/uZgDsVtGwZfPghZGTAli1uOd26wWWXuTC47DI4/3zXa6kxfrMAMCZIu0p38Y/N/+CDHR/w4Y4P2V+5HzgWCBMHTuTygZc3BMLevcfCIDAQYmLcaaILL3TD+PHQv789hWy+fBYAxrRBndaRU5TD8h3LydiRwYc7P+RApTtT2Se+D+P7jmdc33GM7zuetD5pxEfHU1DgXmG5ciWsWOH6IjpyxC0vJcWFwbhxrkuKCy5wzycYczpZABhzCtRpHev3redfO//Fyj0rWblnJXkH3BNkYRLGyKSRjO87nrF9xjKq1yjO6XUOcWE9WLfOhUH9sG3bsWUOGnQsDCwUzOlgAWDMabK/Yj+r9qxqCIRVe1Y1HCWAO1I4J+kcN/RynykRI8nL6c7q1ZCV5Y4SPvvs2DL794dRo+Ccc9wwahSMGAFduviwgabDswAw5kuiquws3UlOUQ45xd5QlMPG4o1U1lQ2TJfSNYURSSMY3nM4I5JG0CdqONV7RvDZuj5s2CDk5MCmTcdOH4m4o4VzznFPLZ911rGhd2+7tmBOzALAGJ/VaR07SnY0hMGm/ZvILc4l9/PchtddAsRHxTOs5zCGJg5lUPchdKsZSm3xEEp3DGFXTh825oSRl3csGADi410QnH22ew/CkCEweLAbkpMtHEKdBYAx7ZSqUlheSO7nuWz63IVC3sE88g7ksaNkx3F9GMVExDA4YTB94/sRL8mEVyZTW5LC4X3J7N+ZQsGWZPZsSYaqbtT30BIbCwMHHguEQYPcS3P69YO+fd2F6chIf7bdfDksAIzpgGrqathVuottB7ax7eA28g7kse3gNgrKCigoL6CwvJDq2uovzBcTHkuPiBTialOIqOzjQqIohZJdKVR9ngKHk+BwL6hIQuqiSE52YdCvn7v+MGDA8UNSkh1FdGQWAMZ0QqrKwaqDFJYXUlhe2BAMDZ/e971leymrLmtyGdHaneiaXoRVJVF7qBdVB3py9FACVPVoGKLqepCc0IP+Z/SgX1IP+if1oF+vOFJShORkdw0iOdk9DGdB0f6cKAAi/CjGGHNqiAiJsYkkxiYyMqnxe5qOd7j6cEMgFFcUU3y4mKLDRRRXBH7msb9iJSVVJcddsK7G9deyK3CB+yNgb3cvJNxn2NEexJJAl7AEukUl0CM6kZ5dEugdn0hyjwT69kygd2IcvRNj6d0zluQzoujeXQizt5P7wgLAmBDRJaoLQxOHMjRxaFDTH6k5QumRUkqqSjhYedB9Vh2ktKqUg5UlFJaUUlhSQtGhEg5UlFJaVULZ0U1U6EE+DztIUVjV8Qss8YaAZyBQgaOxSG0sYXWxRGgs0WFxxEbEERcVS3xMHN1j4+jRNY7E+Fh6dutCt+iuxEfHEx8VT9co971rVFfio+KJi4wjNjKW2IhYYiNjiYmIITIsErHDkiZZABhjmhQdEU2viF706tLku5paVHm0koNVBzlYeZDi8oPsKjpI/v6DHCir5GBZBaUVlRyqqKRMKymvraSippKKoxUcrq7kQE0l+6iAyIMQWQERle4zqhwiq1peeYAwwojxAqFLVByxEbENQREXGee+e20xETHERrjgiImIaQiR+iE6PJroiGiiw6OJCo9q+B4dEU1MREzD8uIi4zpE8FgAGGNOi9hI90e3T3wf976/wa2b/8gRKCyEggLX31JBgeuIr6S0hv2HDnPgcBklFeWUVpVxqKqM8uoyyqsrOVpXCZGVEFEFEZXURVZSEVFJRWQl+yMqiYyrJCK2gvCYCiTqMBL1ORpRgYZXUBdWRY1UUUMlddSe1PaHSRgx4XFESRyRGkdURBRxMWFERYYTJmGESRjhYe57uIQTFxl33NFMfFR8w5FOfHQ8Vw65kv7d+7e84lawADDGtEvR0cfuQjpeBNDdG76oqgoOHoQDB9wQ+L2kBEpL3VCyx/ssOf6zrs5bUFhNQ4i4zyqIOAIRR4iLP0Js12piuh4hpssRorscoVaqKD1cyaHKCiqOVlAXWeEFT4U7egk/ClJLTFwdPRJq6dajlm7d6+gaX0tUdC0VNRXsLNlJWXUZZUfKKKsuo6rm2NHOOze/YwFgjDHNiYlxzzakpLR+XlU4fLg+DCIoKelKaWnXhnAoL4eyMjcc932fm//cXtDLu222Vy83JCW5obgY1qxx75ZY83+Qm3ssbBIT3TRxcZAcC4Ni3fMb0bE1RHYpJyKujJ6He566neSxADDGGI8IdO3qhn79Tv3yL7/82PeKCli/3gVCdrY7UqmsdENFhTtiqayMoLKyB5WVPfjWjFNfjwWAMcb4IC7OvSNi/Hj/arC7b40xJkQFFQAiMllENotInvcC98bjRUR+541fJyJjvPb+IrJcRHJFJEdEfhAwz1wR2SMi2d4w9dRtljHGmJa0eApIRMKBp4ErgHwgU0SWqOrGgMmmAMO8YTzwjPdZA9yvqmtEJB5YLSLLAuZ9UlV/feo2xxhjTLCCOQIYB+Sp6nZVrQYWANMaTTMN+LM6K4AeIpKiqgWqugZAVcuAXKDvKazfGGNMGwUTAH2B3QG/8/niH/EWpxGRgUAqsDKgeY53yuglEUkIsmZjjDGnQDAB0NSzzI27EG12GhHpCiwC7lXV+jdfPAMMAUYDBcC8JlcuMktEskQkq7i4OIhyjTHGBCOYAMgHAh8/6wfsDXYaEYnE/fF/RVXfqJ9AVfepaq2q1gF/wJ1q+gJVfV5V01Q1LSkpKYhyjTHGBCOYAMgEhonIIBGJAm4EljSaZglwm3c30IVAqaoWiOsJ6UUgV1WfCJxBRAKf07sW2NDmrTDGGNNqLd4FpKo1IjIHeBcIB15S1RwRme2NfxZYCkwF8oAK4A5v9ouBW4H1IpLttf0/VV0KPC4io3GninYA3zllW2WMMaZF9kYwY4zp5E70RjB7EtgYY0KUBYAxxoQoCwBjjAlRFgDGGBOiLACMMSZEWQAYY0yIsgAwxpgQZQFgjDEhygLAGGNClAWAMcaEKAsAY4wJURYAxhgToiwAjDEmRFkAGGNMiLIAMMaYEGUBYIwxIcoCwBhjQpQFgDHGhCgLAGOMCVFBBYCITBaRzSKSJyIPNjFeROR33vh1IjKmpXlFJFFElonIVu8z4dRskjHGmGC0GAAiEg48DUwBRgI3icjIRpNNAYZ5wyzgmSDmfRB4X1WHAe97v40xxnxJgjkCGAfkqep2Va0GFgDTGk0zDfizOiuAHiKS0sK804D53vf5wPST3BZjjDGtEBHENH2B3QG/84HxQUzTt4V5e6tqAYCqFohIr6ZWLiKzcEcVAOUisjmImptyBvB5G+c93ay2trHa2sZqa5uOXNuAphqDCQBpok2DnCaYeZulqs8Dz7dmnqaISJaqpp3sck4Hq61trLa2sdrapjPWFswpoHygf8DvfsDeIKdpbt593mkivM+i4Ms2xhhzsoIJgExgmIgMEpEo4EZgSaNplgC3eXcDXQiUeqd3mpt3CTDT+z4TePMkt8UYY0wrtHgKSFVrRGQO8C4QDrykqjkiMtsb/yywFJgK5AEVwB3Nzest+jFgoYjcCewCrj+lW/ZFJ30a6TSy2trGamsbq61tOl1totqqU/LGGGM6CXsS2BhjQpQFgDHGhKiQCICWurLwk4jsEJH1IpItIlk+1/KSiBSJyIaAtnbRZccJapsrInu8fZctIlN9qq2/iCwXkVwRyRGRH3jtvu+7Zmrzfd+JSIyIrBKRT73aHvHa28N+O1Ftvu83r45wEVkrIv/0frdpn3X6awBedxRbgCtwt6VmAjep6kZfC/OIyA4gTVV9f8BERC4DynFPdY/y2h4HDqjqY154Jqjqj9tJbXOBclX99ZddT6PaUoAUVV0jIvHAatyT7bfj875rprZv4vO+ExEBuqhquYhEAv8GfgB8A//324lqm0z7+P/cfUAa0E1Vr27rf6ehcAQQTFcWBlDVfwEHGjW3iy47TlBbu6CqBaq6xvteBuTinoL3fd81U5vvvK5jyr2fkd6gtI/9dqLafCci/YCrgBcCmtu0z0IhAE7UTUV7ocB7IrLa6/aivTmuyw6gyS47fDRHXA+0L/l1eiqQiAwEUoGVtLN916g2aAf7zjuVkY17EHSZqrab/XaC2sD//fYb4EdAXUBbm/ZZKATASXdHcZpdrKpjcD2m3uOd6jDBeQYYAowGCoB5fhYjIl2BRcC9qnrIz1oaa6K2drHvVLVWVUfjegkYJyKj/KijKSeozdf9JiJXA0WquvpULC8UAiCYrix8o6p7vc8iYDHulFV70m677FDVfd5/pHXAH/Bx33nniRcBr6jqG15zu9h3TdXWnvadV08JkIE7x94u9lu9wNrawX67GLjGu3a4APiKiLxMG/dZKARAMF1Z+EJEungX5hCRLsDXgA3Nz/Wla7dddtT/H95zLT7tO++C4YtArqo+ETDK9313otraw74TkSQR6eF9jwW+Cmyifey3Jmvze7+p6k9UtZ+qDsT9LftAVW+hrftMVTv9gOumYguwDfip3/UE1DUY+NQbcvyuDXgNd1h7FHfkdCfQE/fCnq3eZ2I7qu0vwHpgnfcfQIpPtV2CO624Dsj2hqntYd81U5vv+w44D1jr1bABeMhrbw/77US1+b7fAmpMB/55Mvus098GaowxpmmhcArIGGNMEywAjDEmRFkAGGNMiLIAMMaYEGUBYIwxIcoCwBhjQpQFgDHGhKj/Dy35I2s4TtRzAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_metrics('wine_type_loss', 'Wine Type Loss', ylim=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "uYV9AOAMwI9p"
   },
   "source": [
    "### Plots for Confusion Matrix\n",
    "\n",
    "Plot the confusion matrices for wine type. You can see that the model performs well for prediction of wine type from the confusion matrix and the loss metrics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "C3hvTYxIaf3n"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATgAAAEGCAYAAADxD4m3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAb2klEQVR4nO3de5wV5Z3n8c+X5tI2FwW5bCOoGIkGzGiUIVF3HKNmwMQNxtUNxmR5Jc6azGribLJmdHdjZsyScTMxM67KOARNmDVKMN5woiCirJpVAYk3UKRHDHRAWhBjBMOl+7d/nGo9YvfpKjmHc0719+2rXqfqOVVPPU3Lj+dS9TyKCMzM8qhPtQtgZlYpDnBmllsOcGaWWw5wZpZbDnBmllt9q12AYsOHNcThY/tVuxiWwUvPNlW7CJbBH9jOrtipfcljyicHxtbX21Od+9SzOxdFxNR9ud++qKkAd/jYfixbNLbaxbAMpow+rtpFsAyejCX7nMfW19tZtujQVOc2NK8dvs833Ac1FeDMrPYF0EFHtYuRigOcmWUSBLsjXRO12jzIYGaZdaT8rxRJR0l6umh7U9JfShomabGktcnn0KJrrpDUImmNpCk9ldMBzswyCYL2SLeVzCdiTUQcFxHHAScAO4C7gMuBJRExHliSHCNpAjAdmAhMBWZJaih1Dwc4M8usg0i1ZXA68K8R8RtgGjA3SZ8LnJ3sTwPmRcTOiFgHtACTS2XqPjgzyySA9vTBa7ikFUXHsyNidhfnTQduS/ZHRcQmgIjYJGlkkn4I8ETRNa1JWrcc4Mwsswy1sy0RManUCZL6A58Frughr66e3ytZEAc4M8skgN3lnWbtTGBlRGxOjjdLak5qb81AW5LeChQ/KDsG2FgqY/fBmVkmQdCeckvpfN5tngIsAGYk+zOAe4rSp0saIGkcMB5YVipj1+DMLJuA9jJV4CQ1AZ8CvlqUfDUwX9KFwHrgPICIWCVpPrAa2ANcHFH6gTwHODPLpPAmQ5nyitgBHLxX2lYKo6pdnT8TmJk2fwc4M8tItHfZ3197HODMLJPCIIMDnJnlUOE5OAc4M8upDtfgzCyPXIMzs9wKRHudPELrAGdmmbmJama5FIhdUXKWoprhAGdmmRQe9HUT1cxyyoMMZpZLEaI9XIMzs5zqcA3OzPKoMMhQH6GjPkppZjXDgwxmlmvtfg7OzPLIbzKYWa51eBTVzPKo8LK9A5yZ5VAgdvtVLTPLowj8oK+Z5ZXq5kHf+gjDZlYzgkINLs3WE0kHSfqFpBclvSDpREnDJC2WtDb5HFp0/hWSWiStkTSlp/wd4Mwss3b6pNpSuBZYGBFHA8cCLwCXA0siYjywJDlG0gRgOjARmArMklSyM9ABzswyCURHpNtKkTQEOAW4CSAidkXEG8A0YG5y2lzg7GR/GjAvInZGxDqgBZhc6h7ugzOzTArLBqYOHcMlrSg6nh0Rs5P9I4DXgJ9IOhZ4CrgUGBURmwAiYpOkkcn5hwBPFOXVmqR1ywHOzDLKtPDzloiY1M13fYHjga9HxJOSriVpjnZ74/eLUjd3E9XMMgkKbzKk2XrQCrRGxJPJ8S8oBLzNkpoBks+2ovPHFl0/BthY6gYOcGaWWXtSi+tpKyUiXgU2SDoqSTodWA0sAGYkaTOAe5L9BcB0SQMkjQPGA8tK3cNNVDPLJELlfBf168DPJPUHXga+TKHiNV/ShcB64LzCfWOVpPkUguAe4OKIaC+VuQOcmWVSGGQoz6taEfE00FUf3endnD8TmJk2fwc4M8vIazKYWU4VBhnq41UtBzgzy8zTJZlZLnW+yVAPHODMLDMvOmNmuRQBuzsc4MwshwpNVAc4M8upDO+iVpUD3D7a0DKA73/t8HeOX13fny9d9ipbN/XjicVD6Nc/aD5sJ9/6+w0MOrCdF3/dxLWXFV6nC+BL33qVk8/8XXUKb+8xYvQuLrt2PUNH7iE64L5bDubum0ZUu1g1x4+JJCRNpTChXQMwJyKuruT9qmHskTv5xwfXANDeDhccP5GTz3yD1pZGvvLfNtLQF+b8z2bmXTeSP/8fmzj8qLe5fuEaGvrC1s19+YszjuITn/odDf6npura94jZV42m5bkmDhjYzvULX2LlI4NZv7ax2kWrMfXTRK1YKZOZNm8AzgQmAOcnM3Lm1tOPDqb5sJ2MGrObE079/TtB6yMn7GDLpn4ANDbFO+m7d/ZB9fEPYa/wels/Wp5rAuDt7Q1saGlkePPuKpeqNnUk6zL0tFVbJesNk4GWiHgZQNI8CjNyrq7gPatq6T0HcerZb7wvfdFtw/jTae+mv7iyiWu+OZa21v58+7r1rr3VoFFjdvGhY97mxZVN1S5KzSmMotbHsoGVrGceAmwoOu5y9k1JF0laIWnFa1tLTgxQ03bvEk88cCCn/Lv3Brhbrx1FQ9/gtHO2vZN29PE7+PHSNVx3/0vMu24ku/5Q/X/p7F2NTe18Z84r3HjlaHa8VR9/kfenck1Zvj9UMsClmn0zImZHxKSImDTi4Pr9n2n5Q4M58qM7GDpizztpi+cPZdmDQ/ir63/TZVP00PE7aWzq4JU17uOpFQ19g+/MeYWH7hzKr+4/qNrFqVluon6A2Tfr2dK7h76nebr84cHMv2EUf3fnWhqb3o3rr67vz4jRu2joC5tb+9H6r42MGrOrGkW29wm+ec0GNqxt5M7ZHj3tjkdRC5YD45OZN39LYbmvL1TwflXzhx1i5aODufQH77bIb/jvY9i9U1zx+SMBOPqE7Vz6v1p5ftlAfn79OPr2hT59gq9/v5UDD67fpnmeTJy8nTPO28bLqxuZtbgwMv6Tv21m+UNDqlyy2lMvo6gVC3ARsUfSJcAiCo+J3BwRqyp1v2pqbAp+ser596T99P+90OW5Z5y7jTPO3dbld1Zdq5YNYsroY6tdjJoXIfb09gAHEBH3AfdV8h5mtv+5iWpmueQ+ODPLNQc4M8uleprwsj56Cs2sppTrOThJr0h6TtLTklYkacMkLZa0NvkcWnT+FZJaJK2RNKWn/B3gzCyTCNjT0SfVltInI+K4iOhcPvByYElEjAeWJMck77JPByYCU4FZyTvv3XKAM7PMKvyq1jRgbrI/Fzi7KH1eROyMiHVAC4V33rvlAGdmmZT5XdQAHpD0lKSLkrRREbEJIPkcmaSner+9mAcZzCyzSF87G97Zt5aYHRGzi45PjoiNkkYCiyW9WCKvVO+3F3OAM7PMMrxIv6Wob+19ImJj8tkm6S4KTc7NkpojYpOkZqAtOT3z++1uoppZJhHl6YOTNFDS4M594M+A54EFwIzktBnAPcn+AmC6pAHJO+7jgWWl7uEanJllJNrLs2zgKOAuFeYS6wvcGhELJS0H5ku6EFgPnAcQEaskzacwae4e4OKIKDlThQOcmWWWoQ+uRB7xMvC+2Q0iYitwejfXzARmpr2HA5yZZeJ3Uc0sv6LQD1cPHODMLLNamI48DQc4M8skyjfIUHEOcGaWmZuoZpZb5RhF3R8c4MwskwgHODPLMT8mYma55T44M8ulQHR4FNXM8qpOKnAOcGaWkQcZzCzX6qQK5wBnZpnVfQ1O0nWUiNMR8Y2KlMjMaloAHR11HuCAFSW+M7PeKoB6r8FFxNziY0kDI2J75YtkZrWuXp6D6/FhFkknSloNvJAcHytpVsVLZma1K1JuVZbmab1/AKYAWwEi4hnglEoWysxqmYhIt1VbqlHUiNiQLAzRqeRCD2aWczVQO0sjTYDbIOkkICT1B75B0lw1s14oIOpkFDVNE/VrwMXAIcBvgeOSYzPrtZRyq64eA1xEbImICyJiVESMiIgvJst6mVlvVcZBBkkNkn4t6V+S42GSFktam3wOLTr3CkktktZImtJT3mlGUY+QdK+k1yS1SbpH0hHpim5muVTeUdRLeW+31+XAkogYDyxJjpE0AZgOTASmArMkNZTKOE0T9VZgPtAMjAZuB25LXXQzy5fOB33TbD2QNAb4DDCnKHka0Pkc7lzg7KL0eRGxMyLWAS3A5FL5pwlwioj/ExF7ku0W6mYMxcwqISLdBgyXtKJou2ivrP4B+DbQUZQ2KiI2Fe4Tm4CRSfohwIai81qTtG6Vehd1WLL7sKTLgXkUAtvngV+WytTMci79KOqWiJjU1ReSzgLaIuIpSaemyKurm5asbJV6TOSp5OLOTL+6V6bfS1EgM8shlacNdzLwWUmfBhqBIZJuATZLao6ITZKagbbk/FZgbNH1Y4CNpW7QbRM1IsZFxBHJ596bBxnMequ0Aww9BMGIuCIixkTE4RQGDx6KiC8CC4AZyWkzgHuS/QXAdEkDJI0DxgPLSt0j1ZsMko4BJlCIsp2F++c015pZ3qQbQNgHVwPzJV0IrAfOA4iIVZLmA6uBPcDFEVHyraoeA5yk7wKnUghw9wFnAo8BDnBmvVWZhxkjYimwNNnfCpzezXkzgZlp800zinpucrNXI+LLwLHAgLQ3MLMc6ki5VVmaJurbEdEhaY+kIRQ6/NwHZ9Zb5WHCyyIrJB0E/JjCyOpb9NCxZ2b5VqZR1IrrMcBFxH9Odm+UtBAYEhHPVrZYZlbT6j3ASTq+1HcRsbIyRTIzK49SNbhrSnwXwGllLgsvPdvElNHHlTtbq6C2S06qdhEsgz0/f6Is+dR9EzUiPrk/C2JmdSLI8qpWVXnhZzPLrt5rcGZm3an7JqqZWbfqJMClmdFXkr4o6crk+FBJJSeZM7Ocy9G6qLOAE4Hzk+PfAzdUrERmVtMU6bdqS9NE/XhEHC/p1wARsS1ZPtDMeqscjaLuThZ2CABJI6iJ12jNrFpqoXaWRpom6v8G7gJGSppJYaqk71e0VGZW2+qkDy7Nu6g/k/QUhSmTBJwdEV7Z3qy3qpH+tTTSTHh5KLADuLc4LSLWV7JgZlbD8hLgKKyg1bn4TCMwDlhDYfFVM+uFVCe98GmaqB8tPk5mGflqN6ebmdWMzG8yRMRKSX9cicKYWZ3ISxNV0jeLDvsAxwOvVaxEZlbb6miQIc1jIoOLtgEU+uSmVbJQZlbjyvCYiKRGScskPSNplaS/SdKHSVosaW3yObTomisktUhaI2lKT8UsWYNLHvAdFBGX9ZSRmfUi5anB7QROi4i3JPUDHpN0P3AOsCQirpZ0OXA58FeSJlBYIHoiMBp4UNKHS62N2m0NTlLf5MJupy43s95HFEZR02ylRMFbyWG/ZAsKLcS5Sfpc4OxkfxowLyJ2RsQ6oAUoOfFHqRrcMgrB7WlJC4Dbge1FhbuzdPHNLJey9cENl7Si6Hh2RMzuPEhaiU8BRwI3RMSTkkZFxCaAiNgkaWRy+iFA8ZzrrUlat9KMog4DtlJYg6HzebgAHODMeqv0AW5LREzqNptCK/G4ZGnSuyQdUyKvrt7wL1mSUgFuZDKC+jzvBrZUmZpZzpU5AkTEG5KWAlOBzZKak9pbM4XF5qFQYxtbdNkYYGOpfEuNojYAg5JtcNF+52ZmvVQ55oOTNCKpuSHpAOAM4EVgATAjOW0GcE+yvwCYLmmApHHAeHpYhL5UDW5TRFzVw89pZr1ReWpwzcDcpB+uDzA/Iv5F0uPAfEkXAuuB8wAiYpWk+cBqYA9wcakRVCgd4OpjRjsz27+iPO+iRsSzwMe6SN9KYfairq6ZCcxMe49SAa7LG5iZ1UsvfKmFn1/fnwUxs/pRL69qedlAM8vOAc7McqlGpiNPwwHOzDIRbqKaWY45wJlZfjnAmVluOcCZWS7V0Yy+DnBmlp0DnJnlVW6WDTQz25ubqGaWT37Q18xyzQHOzPLIbzKYWa6poz4inAOcmWXjPjgzyzM3Uc0svxzgzCyvXIMzs/yqkwBXal1UM7P3S1bVSrOVImmspIclvSBplaRLk/RhkhZLWpt8Di265gpJLZLWSJrSU1Ed4Mwsk87n4PZ14WcKa5t+KyI+AnwCuFjSBOByYElEjAeWJMck300HJgJTgVnJmqrdcoAzs+wi0m0ls4hNEbEy2f898AJwCDANmJucNhc4O9mfBsyLiJ0RsQ5oASaXuocDnJlllqEGN1zSiqLtoi7zkw6nsAj0k8CoiNgEhSAIjExOOwTYUHRZa5LWLQ8yVMiI0bu47Nr1DB25h+iA+245mLtvGlHtYhkwavBbfO+zSzh44A4ixB1PT+C25X8EwPRJz/H5E56jvaMPj7YcxrUPnwjAV05cybRjX6AjxA8e+Lc8vu7Qav4I1ZXtQd8tETGp1AmSBgF3AH8ZEW9K6vbUbkrTrYoFOEk3A2cBbRFxTKXuU6va94jZV42m5bkmDhjYzvULX2LlI4NZv7ax2kXr9do7xI8ePIkXN4+gqf8ubv3yL3hy3RiGDXybU8ev4z/M+Ty72xsY2rQDgCOGv86UCS2c++PpjBi0nRu/cC9n33g+HdF7G0Dlmg9OUj8Kwe1nEXFnkrxZUnNEbJLUDLQl6a3A2KLLxwAbS+Vfyd/QTyl0BPZKr7f1o+W5JgDe3t7AhpZGhjfvrnKpDGDL9oG8uLlQm96xqz/rtg5lxKDtnHf8Kn7y+PHsbi/0W2/bUfj9nTr+FRatPpLd7Q1s/N0QNmw7kGNGt3Wbf29QplFUATcBL0TEj4q+WgDMSPZnAPcUpU+XNEDSOGA8sKzUPSoW4CLiEeD1SuVfT0aN2cWHjnmbF1c2VbsotpfmA9/kqFFbeH7jKA4b9gYfG7uRf55xB3O+eDcTmgtBbMTg7bz65qB3rml7cyAjB2+vVpGrLyjLIANwMvAl4DRJTyfbp4GrgU9JWgt8KjkmIlYB84HVwELg4ohoL3WDqvfBJZ2OFwE0kr8A0NjUznfmvMKNV45mx1slR7RtPzug325+eM4ifvjgyWzf1Z+GPh0MadzFf5x7DhOb2/jB5x7grFkXoC6ed+j5726+leNNhoh4jK771QBO7+aamcDMtPeoeidCRMyOiEkRMakfA6pdnLJq6Bt8Z84rPHTnUH51/0HVLo4V6dunnR/++0Xcv+rDPLTmCAA2vzmIJWvGAWLVplF0hBja9Afa3hzEvxny1jvXjhyyndfeGlilkteISLlVWdUDXH4F37xmAxvWNnLnbI+e1pbgu59ZyrotB3HLsmPfSV360jgmH/5bAA4d9gb9GtrZtqORpWsPZ8qEFvo1tDP6wDc5dOgbPL9xZHeZ514ZH/StuKo3UfNq4uTtnHHeNl5e3cisxWsA+MnfNrP8oSFVLpkdN+ZVzvroS7zUNox5F84H4PqlH+fuZ47mr896mNv/0zx2tzdw5b2nAeLlLcN44IUPccdF82jvEFcv+pNePYJKRN1MeKmoUGeCpNuAU4HhwGbguxFxU6lrhmhYfFxdNr2tRrVdclK1i2AZrP35j9ixeUO3D5qlMfigMfGxUy5Nde6j9377qZ6eg6ukitXgIuL8SuVtZtVVC83PNNxENbNsAqiTJqoDnJllVx/xzQHOzLJzE9XMcqteRlEd4Mwsmxp5iDcNBzgzy6TwoG99RDgHODPLrkzTJVWaA5yZZeYanJnlk/vgzCy/6uddVAc4M8vOTVQzy6Uo35oMleYAZ2bZuQZnZrlVH/HNAc7MslNHfbRRHeDMLJugbh707cXzLpvZByECRbqtx7ykmyW1SXq+KG2YpMWS1iafQ4u+u0JSi6Q1kqb0lL8DnJllV551UaHrBeIvB5ZExHhgSXKMpAnAdGBics0sSSXX4nSAM7PsyhTgulkgfhowN9mfC5xdlD4vInZGxDqgBZhcKn8HODPLprMPLs0GwyWtKNouSnGHURGxCSD57Fyj8RBgQ9F5rUlatzzIYGaZZRhF3VLGVbW6Wg2sZDXRNTgzyyhl8/SDPwy8WVIzQPLZlqS3AmOLzhsDbCyVkQOcmWUTVDrALQBmJPszgHuK0qdLGiBpHDAeWFYqIzdRzSy7Mj0HV7xAvKRW4LvA1cB8SRcC64HzACJilaT5wGpgD3BxRLSXyt8BzswyK9eElyUWiD+9m/NnAjPT5u8AZ2bZ+WV7M8ulCGivj3e1HODMLDvX4MwstxzgzCyXAvCaDGaWTwHhPjgzy6PAgwxmlmPugzOz3HKAM7N82qf3TPcrBzgzyyYALzpjZrnlGpyZ5ZNf1TKzvAoIPwdnZrnlNxnMLLfcB2dmuRThUVQzyzHX4Mwsn4JoL7kUQs1wgDOzbDxdkpnlmh8TMbM8CiBcgzOzXApPeGlmOVYvgwyKGhrulfQa8Jtql6MChgNbql0IyySvv7PDImLEvmQgaSGFP580tkTE1H25376oqQCXV5JWRMSkapfD0vPvLB/6VLsAZmaV4gBnZrnlALd/zK52ASwz/85ywH1wZpZbrsGZWW45wJlZbjnAVZCkqZLWSGqRdHm1y2M9k3SzpDZJz1e7LLbvHOAqRFIDcANwJjABOF/ShOqWylL4KVC1B1OtvBzgKmcy0BIRL0fELmAeMK3KZbIeRMQjwOvVLoeVhwNc5RwCbCg6bk3SzGw/cYCrHHWR5mdyzPYjB7jKaQXGFh2PATZWqSxmvZIDXOUsB8ZLGiepPzAdWFDlMpn1Kg5wFRIRe4BLgEXAC8D8iFhV3VJZTyTdBjwOHCWpVdKF1S6TfXB+VcvMcss1ODPLLQc4M8stBzgzyy0HODPLLQc4M8stB7g6Iqld0tOSnpd0u6Smfcjrp5LOTfbnlJoIQNKpkk76APd4RdL7Vl/qLn2vc97KeK+/lvRfs5bR8s0Brr68HRHHRcQxwC7ga8VfJjOYZBYRfx4Rq0ucciqQOcCZVZsDXP16FDgyqV09LOlW4DlJDZL+TtJySc9K+iqACq6XtFrSL4GRnRlJWippUrI/VdJKSc9IWiLpcAqB9L8ktcc/kTRC0h3JPZZLOjm59mBJD0j6taR/ouv3cd9D0t2SnpK0StJFe313TVKWJZJGJGkfkrQwueZRSUeX4w/T8skr29chSX0pzDO3MEmaDBwTEeuSIPG7iPhjSQOAX0l6APgYcBTwUWAUsBq4ea98RwA/Bk5J8hoWEa9LuhF4KyJ+mJx3K/D3EfGYpEMpvK3xEeC7wGMRcZWkzwDvCVjd+EpyjwOA5ZLuiIitwEBgZUR8S9KVSd6XUFgM5msRsVbSx4FZwGkf4I/RegEHuPpygKSnk/1HgZsoNB2XRcS6JP3PgD/q7F8DDgTGA6cAt0VEO7BR0kNd5P8J4JHOvCKiu3nRzgAmSO9U0IZIGpzc45zk2l9K2pbiZ/qGpM8l+2OTsm4FOoCfJ+m3AHdKGpT8vLcX3XtAintYL+UAV1/ejojjihOSv+jbi5OAr0fEor3O+zQ9T9ekFOdAoWvjxIh4u4uypH73T9KpFILliRGxQ9JSoLGb0yO57xt7/xmYdcd9cPmzCPgLSf0AJH1Y0kDgEWB60kfXDHyyi2sfB/5U0rjk2mFJ+u+BwUXnPUChuUhyXmfAeQS4IEk7ExjaQ1kPBLYlwe1oCjXITn2AzlroFyg0fd8E1kk6L7mHJB3bwz2sF3OAy585FPrXViYLp/wThZr6XcBa4DngH4H/u/eFEfEahX6zOyU9w7tNxHuBz3UOMgDfACYlgxireXc092+AUyStpNBUXt9DWRcCfSU9C3wPeKLou+3ARElPUehjuypJvwC4MCnfKjwNvJXg2UTMLLdcgzOz3HKAM7PccoAzs9xygDOz3HKAM7PccoAzs9xygDOz3Pr/LCZt+N6MYuYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_confusion_matrix(test_Y[1], np.round(type_pred), title='Wine Type', labels = [0, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "GW91ym8P2I5y"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQwAAAEWCAYAAACTwaluAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAcEElEQVR4nO3df5xcdX3v8debZK0bii4YqGQxgBXTikiCKz/k1mL9kQfKj0h5aKxcK7cFpVy0XJvbptf663r1+ojXWuReFEEE1GjBkFIvGvVWEW1B8oMQENLyI0A2gOFHEglb2CSf+8ecgdnJzM6Z3Tlz5px5Px+PfezMd86cfIZl33u+53zP96uIwMwsjX3yLsDMisOBYWapOTDMLDUHhpml5sAws9QcGGaWmgPDzFJzYFhqkp6q+dojaazm+Xvyrs+yJw/csqmQtAn404j4Ud61WPf4CMOmRdJvSHpC0lE1bQclRx8HSjpJ0mZJfy3pMUmbao9Gkvd/TtKDkh6V9CVJg/l8GmvFgWHTEhHPAN8Czqppfjfwo4jYmjx/KTAbGAb+GLhU0rzktc8CrwTmA69ItvloF0q3KXBgWCdcCfyRpOr/T/8RuLpum7+JiGci4kbg/wLvlCTgHODCiHgiIn4NfBpY3K3CrT0z8y7Aii8ibpG0E/h9SQ9TOVK4vmaTJyNiZ83zB4A5wIHALGBNJTsAEDAj+6ptKhwY1ilXUumWPAJcGxH/XvPa/pL2rQmNucAdwGPAGHBkRIx2tVqbEndJrFOuBt5BJTSuavD6JyS9QNLvAacA10TEHuArwN9KOghA0rCkhd0q2trjwLCOiIjNwFoggJvqXn4EeBLYAnwD+EBE3J289pfAPcDNknYAPwLmYT3J4zCsYyR9FdgSER+paTsJ+HpEHJJbYdYxPodhHSHpMOAMYEG+lViW3CWxaZP036mcxFwWEffnXY9lx10SM0vNRxhmllrhzmHMnj07DjvssLzLMGvbngg2PfY0O5/dxdwDZvHiwYG8S2pqzZo1j0XEgfXthQuMww47jNWrV+ddhllbdj6zi7OvuJXtDz7Jle+az6lHz8m7pElJeqBRu7skZhmrhsWaB5/kCwUIi8k4MMwyVKawAAeGWWbKFhbgwDDLRBnDAhwYZh1X1rCAAl4lqbVy3SjLVm1ky7Yx5gwNsmThPBYtGM67LOtjZQ4LKHBgrFw3ytIVGxgb3w3A6LYxlq7YAODQsFyUPSygwF2SZas2PhcWVWPju1m2amNOFVk/64ewgAIHxpZtY221m2WlX8ICChwYc4Yaz0TfrN0sC/0UFlDgwFiycB6DAxPnih0cmMGShZ6sybqj38ICCnzSs3pi01dJLA/9GBZQ4MCASmg4IKzb+jUsoMBdErM89HNYgAPDLLV+DwtwYJil4rCocGCYteCweJ4Dw2wSDouJMg0MSRdKulPSHZKWS3ph3euSdJGkeyTdLumYLOsxa4fDYm+ZBYakYeCDwEhEvJrKityL6zY7GTgi+ToXuCSresza4bBoLOsuyUxgUNJMYBaVtTVrnQ5cFRU3A0OSDs64JrNJOSyayywwImIU+BzwIPAwsD0iflC32TDwUM3zzUnbBJLOlbRa0uqtW7dmVbKZw6KFLLsk+1M5gjgcmAPsK+ms+s0avHWvpdgi4tKIGImIkQMP3GupBLOOcFi0lmWX5M3A/RGxNSLGgRXA6+u22Qy8rOb5IezdbTHLnMMinSwD40HgeEmzJAl4E3BX3TbXA+9NrpYcT6Xb8nCGNZntxWGRXmY3n0XELZKuBdYCu4B1wKWSPpC8/iXgBuBtwD3A08DZWdVj1ojDoj2FW719ZGQkvFSidYLDojlJayJipL7dIz2tLzkspsaBYX3HYTF1DgzrKw6L6XFgWN9wWEyfA8P6gsOiMxwYVnoOi85xYFipOSw6y4FhpeWw6DwHhpWSwyIbDgwrHYdFdhwYVioOi2w5MKw0HBbZc2BYKTgsusOBYYXnsOgeB4YVmsOiuxwYVlgOi+7LchLgeZJuq/naIenP67Y5SdL2mm0+mlU9Vi4Oi3xkOUXfRmA+gKQZwChwXYNNb4qIU7Kqw8rHYZGfzAKjzpuAeyPigU7udOW6UZat2siWbWPMGRpkycJ5LFqw17ImViIOi3x16xzGYmB5k9dOkLRe0vckHdlog0YLGa1cN8rSFRsY3TZGAKPbxli6YgMr141m8wksdw6L/GUeGJJeAJwGXNPg5bXAoRFxNPBFYGWjfTRayGjZqo2Mje+esN3Y+G6WrdrYyfKtRzgsekM3jjBOBtZGxKP1L0TEjoh4Knl8AzAgaXaanW7ZNtZWuxWXw6J3dCMw3k2T7oiklyaLHCHp2KSex9PsdM7QYFvtVkwOi96SaWBImgW8hcoyidW2D1QXMwLOBO6QtB64CFgcKRdKWbJwHoMDMya0DQ7MYMnCeR2p3fLnsOg9mV4liYingZfUtX2p5vHFwMVT2Xf1aoivkvSOTl61clj0pm5dVs3EogXDDogeUb1qVT0RXb1qBbT9M3JY9C4PDbeO6NRVK4dFbyv0EUZZlGEAWieuWjksep+PMHJWlgFo071q5bAoBgdGzsoyAG3JwnkM7KMJbQP7KNVVK4dFcTgwclaqAWhq8bwBh0WxODByVpYBaMtWbWR898QhNOO7Y9IjJYdF8TgwclaWAWjtHik5LIrJV0lyVpYBaC8eHGDb2HjD9noOi+JyYPSAMgxAU5PzFfXtDotic5fEOmLb03sfXdS3OyyKz4FhHdHq5K3DohwK3SUpwwhJKMfnWLJwHh++Zj279zx/pWRGMg7DYVEehQ2MTt7slKeyfI7VDzwxISwAdu8J/uW+x/jmLQ86LEqisF2SsoyQLMvnWH7LQw3bv33rZodFiRQ2MMoyQrIsn2P3JPMeOSzKI++FjCTpIkn3SLpd0jFp91+WEZJDs/YepzBZe6+a0eS6qsBhUSKZBUZEbIyI+RExH3gt8DR7L2R0MnBE8nUucEna/ZdlhORT/974cmSz9l717uNe1rD9PcfP7XIllqW8FzI6HbgqmcfzZklDkg6OiIdb7bAsIyTH97TX3qs+tegoxnfv4du3bgYqRxbvOX4un1p0VL6FWUd1KzCaLWQ0DNSeLductE0IDEnnUjkCYe7c5/9ilWGEZFnsfGYXt2568rnnc4YGGTn0gBwrsizkvZBRo47vXmfPGi1kZL1j5zO7OPXin3Hf1p3PtRV1IiCbXK4LGVE5oqjt/B4CbOlCTT1j1kDjH0Gz9l5THZRVGxZVRbw8bJPLdSEj4HrgvcnVkuOB7WnOX5TJp894DXUTVbGPKu29rnYEZzOjBbs8bJPLeyGjG4D7gHuArwB/lmU9vWjRgmE+/875DA8NImB4aJDPv3N+z5+bqR/uXR96Vc3arZjyXsgogPOzrKEIinbyttG9IRcsX9dw2z2p1rGzoihGR9l6hm8k62+FvfmsTIpyt6rDwhwYOSvK3aoOC4OCB0ZR/jJPZrK7VXvlszgsrKqwgVGUv8ytNLvs2CuXI9OGhWgw4o5US5NYgaQ66SnptyX9RvL4JEkflDSUbWmTK8s8Er18ObKdI4tmF0N8kaRc0l4l+Q6wW9IrgMuBw4FvZlZVCmWZR6LZZce8L0e6G2KNpA2MPRGxC3gH8IWIuBA4OLuyWivLfBi9yGFhzaQNjHFJ7wb+GPhu0pbrDC9lmQ9jsMk9I83as+awsMmk/b/ybOAE4H9ExP2SDge+nl1ZrS1aMMxnzjhqwpDqz5xxVKFOeEJvdUmmExbDTY7smrVbMaW6ShIRvwQ+WPP8fuB/ZlVUWkUbUt3IM7saz5TTrD0r0z2yeOPvHMjXb36wYbuVR9qrJCdK+qGkf5V0n6T7Jd2XdXHWHZ3ohqxYs7mtdiumtOMwLgcuBNYAu1tsawXSqXMWTzeZU7BZuxVT2sDYHhHfy7SSKSjDSM88+QSntSttYPxY0jIq81o8U22MiLWZVJVCWUZ65sVhYVORNjCOS76P1LQF8AedLSe9ItyD0ascFjZVaa+SvHEqO0+Gj18GvJpKwPyniPiXmtdPAv4BuD9pWhERn0yz716/B6NXOSxsOlIFhqQXAx8D3pA03Qh8MiK2t3jr3wHfj4gzk9nDZzXY5qaIOCVtwVUzpIbL8zVbgcscFjZ9aQdufRX4NfDO5GsHcMVkb5D0IioBczlARDwbEdumXupEzdbynGyNz37msLBOSBsYvx0RH4uI+5KvTwAvb/GelwNbgSskrZN0maR9G2x3gqT1kr4n6chGO5J0rqTVklZv3boV8MjCdjgsrFPSBsaYpP9QfSLpRKDVyYKZwDHAJRGxANgJ/FXdNmuBQyPiaOCLwMpGO2q0kFFZ7iXJmsPCOintVZLzgCuTcxkCngDe1+I9m4HNEXFL8vxa6gIjInbUPL5B0v+RNDsiHmtVUFnWVs2Sw8I6Le1VktuAo5PzEhN+0Sd5zyOSHpI0LyI2UlmQ+Ze120h6KfBoRISkY6kc8Tyetvgy3EuSFYeFZWHSwJB0VkR8XdJ/qWsHICI+32L/FwDfSK6Q3AecXV3EKFmf5EzgPEm7qHRxFidrldg0OCwsK62OMKonKfdr8FrLX+zkyGSkrrl2IaOLgYtb7acZDw3fm8PCsjRpYETEl5OHP4qIn9e+lpz4zM3KdaMsuWY948nEEaPbxlhyzXqgf4eGOywsa2mvknwxZVvXfPz6O58Li6rxPcHHr78zp4ry5bCwbmh1DuME4PXAgXXnMV4EzGj8ru7YNjbeVnuZOSysW1qdw3gB8JvJdrXnMXZQOWFpOXNYWDe1OodxI3CjpK9FxANdqslSclhYt6U9h3FZ7cJFkvaXtCqjmlIZGmw8aXmz9rJxWFge0gbG7NobxyLiSeCgbEpK5+OnHclA3fJgA/uIj5/W8HaUUnFYWF5SL2QkaW71iaRDyXkVvEULhjn28P0ntB17+P6lv6TqsLA8pQ2M/wb8TNLVkq4Gfgosza6s1j6ycgM/v/eJCW0/v/cJPrJyQ04VZc9hYXlLFRgR8X0qd55+G/h74LURkes5jG80WANjsvaic1hYL5g0MCT9TvL9GGAusAUYBeYmbbnpp9XCHRbWK1qNw/gwcA7wvxq8luskwP3CYWG9pNU4jHOS71OaBNimx2FhvabV0PAzJns9IlZ0thyr5bCwXtOqS3Jq8v0gKveU/FPy/I3AT6gsbGQZcVhYr2nVJTkbQNJ3gVdFxMPJ84OB/519ef3NYWG9Ju04jMOqYZF4FHhlqzdJGpJ0raS7Jd2V3P1a+7okXSTpHkm3533lpdc4LKzXpJ0E+CfJvSPLqVwdWQz8OMX7Wi1kdDJwRPJ1HHAJzy/LaGY9Ju0kwP9Z0jt4fuWzSyPiusneU7OQ0fuSfTwLPFu32enAVck8njcnRyQH1x3NmFmPSHuEAZU1RH4dET+SNEvSfhHx60m2r13I6GhgDfChiNhZs80w8FDN881J24TAkHQucC7A3LlzMbN8pDqHIekcKuuKVOf4HKbJokM10ixk1Ggh1L0GazZayKgMdj6zK+8SzNqS9qTn+cCJVGbaIiL+jda3tzdayKj+pOZm4GU1zw+hMvy89KqDssyKJG1gPJOcgwBA0kxa3LYREY8AD0mqrl2410JGwPXAe5OrJccD2/vh/EXtCE6zIkkbGDdK+mtgUNJbgGuAf0zxvupCRrcD84FPS/pAdTEj4AYqCxzdA3wF+LO2qi+g+uHeZkWS9qTnXwJ/CmwA3k/lF/2yVm9KsZBRUOnu9IVG94ZcsHxd3mWZpdYyMCTtA9weEa+mchRgU+AbyawMWnZJImIPsL52ij5rj8PCyiJtl+Rg4E5Jv6ByeRSAiDgtk6pKxGFhZZI2MD6RaRUl5bCwsmk1H8YLgQ8Ar6BywvPyiPBooxQcFlZGrc5hXEnlKscGKjeKNZqqz+o4LKysWnVJXhURRwFIuhz4RfYlFZvDwsqs1RHGc0uhuyvSmsPCyq7VEcbRknYkj0VlpOeO5HFExIsyra5AHBbWD1pN0TejW4UUmcPC+kXae0msCYeF9RMHxjQ4LKzfODCmyGFh/ciBMQUOC+tXDow2OSysnzkw2uCwsH7XzqzhbZO0Cfg1sBvYFREjda+fBPwDcH/StCIiPpllTVPlsDDLODASb4yIxyZ5/aaIOKULdUyZw8Kswl2SFhwWZs/LOjAC+IGkNcliRI2cIGm9pO9JOrLRBpLOlbRa0uqtW7dmV20dh4XZRFl3SU6MiC2SDgJ+KOnuiPhpzetrgUMj4ilJb6OyONIR9TuJiEuBSwFGRkYmXd6gUxwWZnvL9AgjIrYk338FXAccW/f6joh4Knl8AzAgaXaWNaXhsDBrLLPAkLSvpP2qj4G3AnfUbfNSSUoeH5vU83hWNaXhsDBrLssuyW8B1yV5MBP4ZkR8v7qIUUR8CTgTOE/SLmAMWJysVZILh4XZ5DILjIi4Dzi6QXvtQkYXAxdnVUM7HBZmrfmyKg4Ls7T6PjAcFmbp9XVgOCw6Z3hosK12K6a+DQyHRWctWTiPwYGJMzoODsxgycJ5OVVkWejGvSQ9x2HReYsWDAOwbNVGtmwbY87QIEsWznuu3cqh7wLDYZGdRQuGHRAl11eB4bDI1sp1oz7CKLm+CQyHRbZWrhtl6YoNjI3vBmB02xhLV2wAcGiUSF+c9HRYZG/Zqo3PhUXV2Phulq3amFNFloXSB4bDoju2bBtrq92KqdSB4bDonjlNxls0a7diKm1gOCy6y+Mw+kMpT3o6LLrP4zD6QykDw2GRD4/DKL9SdkkcFmbZyDQwJG2StEHSbZJWN3hdki6SdI+k2yUd04l/12Fhlo281yU5mcqkv0cAxwGXJN+nxWFhlo28uySnA1dFxc3AkKSDc67JzJrIe12SYeChmuebkzYz60F5r0uiBu/ZaxLgJGzOBZg7d242lZpZS7muS0LliOJlNc8PAbY02M+lETESESMHHnggO5/ZlVXJZjaJXNclAa4H3ptcLTke2B4RD0+23z0RnH3FrZnUnIdZA41/BM3azfKU97okNwBvA+4BngbObrXTTY89zfYHn8ys6G5rtghLbouzmE0i73VJAji/nf3ufHYXV75rPhcsXzf9InvA2PiettrN8lS44965B8zyOAuznBQuMF48OJB3CWZ9q3CBUTb7z2ocgM3azfLkwMjZx049koEZE4ejDMwQHzv1yJwqMmuusLe3S9BonXc1GgrWwzyPhBVJYQPj9S8/gJ/f+0TD9qLxPBJWFIXtkmx6vPHkss3azWz6ChsYnqXarPsKGxjNLq/6sqtZdgobGM/u2t1Wu5lNX2ED4+kmQ6ebtZvZ9BU2MMys+wobGB4hadZ9hQ2Mt7+m8dSfzdrNbPoKGxg/vntrW+1mNn2FDQyPwzDrvswDQ9IMSeskfbfBaydJ2p4sdHSbpI+m3a9XCzfrvm4cYXwIuGuS12+KiPnJ1yfT7tSrhZt1X9ZLJR4CvB24rNP7XrRgmM+ccRTDQ4MIGB4a5DNnHOWbuMwylPXdql8A/iuw3yTbnCBpPZXlBf4iIu6s36DZuiS+y9Osu7JcZuAU4FcRsWaSzdYCh0bE0cAXgZWNNqpfl8TM8pFll+RE4DRJm4BvAX8g6eu1G0TEjoh4Knl8AzAgaXaGNZnZNGQWGBGxNCIOiYjDgMXAP0XEWbXbSHqpkoVLJB2b1PN4VjWZ2fR0fcatuoWMzgTOk7QLGAMWJ2uV9JWV60Y9RZ8Vgor2+zkyMhKrV6/Ou4yOWblulKUrNjA2/vxt+YMDM3zFx3IlaU1EjNS3F3akZ1ksW7VxQlgAjI3vZtmqjTlVZNacAyNnHuJuRVLYWcOhHH3/OUODjDYIBw9xt15U2COMat9/dNsYAYxuG2Ppig2sXDead2lt8RB3K5LCBkZZ+v4e4m5FUtguSZn6/h7ibkVR2CMM395u1n2FDQz3/c26r7BdEi9ibNZ9hQ0McN/frNsK2yUxs+5zYJhZag4MM0vNgWFmqTkwzCy1ws2HIWkr8EBd82zgsRzK6TR/jt5Rhs8AU/8ch0bEXhPoFi4wGpG0utFkH0Xjz9E7yvAZoPOfw10SM0vNgWFmqZUlMC7Nu4AO8efoHWX4DNDhz1GKcxhm1h1lOcIwsy5wYJhZaoUPDEmbJG2QdJukQi5YImlI0rWS7pZ0l6QT8q6pXZLmJT+D6tcOSX+ed11TIelCSXdKukPSckkvzLumqZD0oeQz3Nmpn0Xhz2Eka7eORERhB9lIuhK4KSIuk/QCYFZEbMu7rqmSNAMYBY6LiPpBdj1N0jDwM+BVETEm6e+BGyLia/lW1h5Jr6aypvGxwLPA94HzIuLfprPfwh9hFJ2kFwFvAC4HiIhnixwWiTcB9xYtLGrMBAYlzQRmAVtyrmcqfhe4OSKejohdwI3AO6a70zIERgA/kLRG0rl5FzMFLwe2AldIWifpMkn75l3UNC0GluddxFRExCjwOeBB4GFge0T8IN+qpuQO4A2SXiJpFvA24GXT3WkZAuPEiDgGOBk4X9Ib8i6oTTOBY4BLImIBsBP4q3xLmrqkS3UacE3etUyFpP2B04HDgTnAvpLOyreq9kXEXcBngR9S6Y6sB3ZNd7+FD4yI2JJ8/xVwHZU+W5FsBjZHxC3J82upBEhRnQysjYhH8y5kit4M3B8RWyNiHFgBvD7nmqYkIi6PiGMi4g3AE8C0zl9AwQND0r6S9qs+Bt5K5VCsMCLiEeAhSdXpzt8E/DLHkqbr3RS0O5J4EDhe0ixJovLzuCvnmqZE0kHJ97nAGXTg51LoSYCB3wKuq/xcmQl8MyK+n29JU3IB8I3kcP4+4Oyc65mSpK/8FuD9edcyVRFxi6RrgbVUDuHXUdxh4t+R9BJgHDg/Ip6c7g4Lf1nVzLqn0F0SM+suB4aZpebAMLPUHBhmlpoDw8xSc2D0mWSocPWO0kckjdY8f0EH9v9xSZ+pa5svqelYhuQ9fzHdf9uyV/RxGNamiHgcmA+VX1TgqYj4XPV1STOTm5WmajnwPWBpTdti4JvT2Kf1CB9hGJK+Junzkn4MfLb+L34yp8JhyeOzJP0iOSL5cnIr+3MiYiOwTdJxNc3vBL4l6RxJt0paL+k7yUCv+lp+ImkkeTw7mb4ASTMkLUvef7uk9yftB0v6aVLPHZJ+r6P/cWwCB4ZVvRJ4c0R8uNkGkn4XeBeVG/7mA7uB9zTYdDmVowokHQ88nszDsCIiXhcRR1MZbv0nbdT3J1TuHH0d8DrgHEmHA38ErErqORq4rY19WpvcJbGqayJid4tt3gS8Frg1GY4/CPyqwXbfAv5Z0oeZeKv7qyV9ChgCfhNY1UZ9bwVeI+nM5PmLgSOAW4GvShoAVkaEAyNDDgyr2lnzeBcTjz6rU9QJuDIias9P7CUiHkq6Er8P/CFQnXLwa8CiiFgv6X3ASQ3eXvtv106NJ+CCiNgrZJIpDd4OXC1pWURcNVl9NnXuklgjm0husZd0DJW5IQD+H3BmzV2QB0g6tMk+lgN/S2Xmrc1J237Aw8nRQKOuTPXffm3y+Mya9lXAecl7kfTK5G7lQ4FfRcRXqMxaVuSpAXqeA8Ma+Q5wgKTbgPOAfwWIiF8CH6Eyw9ntVCZnObjJPq4BjqTSPan6G+CW5H13N3nf56gEwz9TWUi46jIqt/2vlXQH8GUqR8gnAbdJWkflaObv2vqk1hbfrWpmqfkIw8xSc2CYWWoODDNLzYFhZqk5MMwsNQeGmaXmwDCz1P4/1XUCs7xOO/IAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "scatter_plot = plot_diff(test_Y[0], quality_pred, title='Type')"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "include_colab_link": true,
   "name": "exercise-answer.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}